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United  States  General  Accounting  Office 
Washington,  D.C.  20548 


National  Security  and 
International  Affairs  Division 


B-280524 

September  10, 1999 

The  Honorable  James  M.  Inhofe 
Chairman,  Subcommittee  on  Readiness 
and  Management  Support 
Committee  on  Armed  Services 
United  States  Senate 

Dear  Mr.  Chairman: 

In  recent  years,  the  Navy  has  implemented  many  changes  aimed  at  making 
its  fleet  support  activities  more  efficient  and  effective.  An  underlying 
assumption  was  that  consolidating  similar  activities  within  a  region  could 
eliminate  the  inefficiencies  and  redundancies.  In  1998,  as  part  of  this  effort, 
the  Navy  implemented  a  pilot  project  consolidating  the  management, 
operations,  and  funding  of  the  Pearl  Harbor  Naval  Shipyard  and  the  Naval 
Intermediate  Maintenance  Facility  in  Hawaii.  The  Navy  expects  that  the 
pilot  project,  commonly  called  the  Pearl  Harbor  Pilot,  will  serve  as  a  model 
for  other  potential  consolidations  by  confirming  that  integrating  shipyard 
and  intermediate  activities  can  result  in  greater  efficiency  and  lower  overall 
unit  costs. 

As  you  requested,  we  examined  the  Navy's  progress  in  implementing  this 
pilot  project.  This  report  discusses  (1)  the  preliminary  results  of  the  Pearl 
Harbor  Pilot  on  improving  performance  of  maintenance  activities,  (2)  the 
usefulness  of  the  pilot  as  a  model  for  future  consolidations,  and  (3)  issues 
related  to  financial  and  organizational  structures  for  such  consolidations. 
Our  scope  and  methodology  are  described  in  appendix  I. 


Results  in  Brief 


Although  the  Pearl  Harbor  Pilot  is  not  complete,  preliminaiy  results  have 
been  mbced,  showing  either  improvements,  no  improvements,  or 
insufficient  data  to  determine  results.  Where  data  are  available,  overall 
indications  are  that  the  pilot  has  the  potential  to  improve  maintenance 
activities  in  Hawaii.  While  the  Navy's  pilot  test  plan  calls  for  evaluating 
performance  using  nine  metrics,  data  has  been  gathered  for  only  five  to 
compare  performance  under  the  consolidated  operation  with  performance 
under  the  preconsolidation  organizations.  Preliminary  results  for  two 
metrics  indicate  improvements  that  meet  or  exceed  the  Navy's 
expectations,  and  two  others  indicate  improvements  that  fall  slightly  short 
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of  the  Navy's  expectations,  while  one  metric  indicates  no  improvement. 
Other  positive  results  include  increasing  workforce  flexibility  by 
integrating  nearly  4,000  workers  from  two  work  centers  into  a  single 
workforce.  Consequently  the  number  of  workers  assigned  daily  to  the 
excess  labor  shop  has  dropped  from  about  200  to  below  10.  Furthermore, 
the  Navy  has  reduced  the  maintenance  infrastructure  by  11  buildings 
(114,131  square  feet)  and  plans  to  eliminate  another  6  buildings  (24,907 
square  feet) .  At  the  same  time,  other  anticipated  efficiencies  have  yet  to  be 
realized.  For  example,  not  all  the  industrial  plant  equipment  has  been 
consolidated  and  the  projected  number  of  overhead  workers  has  not  been 
moved  to  direct  maintenance  positions  to  improve  productivity. 

The  Pearl  Harbor  Pilot  is  likely  to  serve  only  as  a  general  model  for  future 
consolidation  efforts  because  of  unique  aspects  of  ship  maintenance 
activities  in  Hawaii,  such  as  the  close  proximity  of  facilities  and  the  large 
portion  of  fleet-funded  work  than  in  other  locations.  Likewise,  because  of 
this  uniqueness,  the  pilot  provides  only  a  general  indication  that  future 
consolidations  elsewhere  will  result  in  similar  efficiencies.  Nevertheless,  it 
does  provide  general  information  on  such  issues  as  combining  workforces 
from  separate  activities  and  consolidating  equipment  and  facilities. 

The  Pearl  Harbor  Pilot  has  sharpened  the  debate  over  the  most  appropriate 
financial  and  organizational  structures  for  such  consolidated  activities. 
Conflicting  views  continue  to  exist  within  the  Departments  of  Defense  and 
the  Navy  over  whether  these  consolidated  activities  should  operate  under 
direct  appropriations,  the  Navy  Working  Capital  Fund,  or  a  combination  of 
the  two  financial  structures,  particularly  in  terms  of  maintaining  visibility 
over  total  operational  costs.  Similarly,  little  progress  has  been  made  toward 
resolving  the  departments'  differences  over  the  most  appropriate 
organizational  structure  to  manage  these  types  of  activities. 

This  report  recommends  that  the  Navy  take  steps  to  address  unresolved 
issues  related  to  financial  and  organizational  structures  as  it  proceeds  with 
similar  consolidations. 


Background 


In  March  1994,  the  Chief  of  Naval  Operations  (CNO)  announced  a  Regional 
Maintenance  Program  to  streamline  the  Navy  ship  repair  and  maintenance 
processes,  reduce  infrastructure  and  costs,  and  maximize  outputs.  The 
program  consisted  of  three  phases:  (1)  optimizing  intermediate-level 
maintenance  through  consolidation  of  intermediate  activities, 

(2)  integrating  intermediate  and  depot  activities  with  management  by  the 
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The  Pearl  Harbor  Pilot 


fleet  commanders,  and  (3)  conducting  fleet  maintenance  using  a  single 
process.  The  first  phase,  optimizing  intermediate-level  maintenance  by 
minimizing  redundant  intermediate  ship  maintenance  capacity  and 
capability,  is  nearing  completion.  To  ensure  the  validity  of  the  second  phase 
of  the  program  and  to  provide  a  model  for  other  potential  consolidations, 
the  Navy  implemented  the  Pearl  Harbor  Pilot  in  1998.  Its  completion  date 
has  not  been  established.  The  third  phase,  performing  a  single  maintenance 
process  by  using  common  business  and  production  processes,  is  underway 
and  scheduled  to  be  completed  in  fiscal  year  2001. 


Prior  to  the  consolidation,  the  Navy  recognized  that  organizational  and 
funding  structures  inhibited  the  ability  of  the  Pearl  Harbor  Naval  Shipyard 
(PHNSY)  and  Naval  Intermediate  Maintenance  Facility  (NIMF)  to  readily 
share  workloads  and  resources.  The  shipyard  was  managed  by  the  Naval 
Sea  Systems  Command  (NAVSEA)  and  funded  through  the  Navy  Working 
Capital  Fund  (NWCF),^  while  the  intermediate  maintenance  facility  was 
managed  by  the  Pacific  Fleet  (PACFLT)  and  financed  through  direct 
appropriations.^  During  the  pilot,  PACFLT  assumed  ownership  and  overall 
management  and  financial  responsibility  for  the  consolidated  activity, 
while  NAVSEA  continued  to  be  the  technical  and  operating  authority.  The 
Navy  named  the  combined  activity  the  Pearl  Harbor  Naval  Shipyard  & 
Intermediate  Maintenance  Facility  (NSY  &  IMF).  To  achieve  a  fully 
integrated  organization,  the  Navy  decided  the  Pearl  Harbor  Pilot  should  use 
a  single  financial  structure  and  selected  direct  appropriations  rather  than 
the  working  capital  fund  as  the  funding  approach.  This  latter  decision  was 
based  on  several  factors,  but  Navy  officials  believed  that  direct 
appropriations  were  better  suited  to  achieve  the  pilot  goals  and  matched 
the  financial  structure  of  the  largest  shipyard  customer-PACFLT. 


*  Under  the  working  capital  fund  arrangement,  activities  such  as  a  shipyard  sell  goods  and 
services  to  customers  based  on  predetermined  rates  designed  to  recoup  the  full  cost  of 
goods  and  services,  including  any  military  personnel  costs.  Customers  pay  for  the  goods  and 
services  with  their  operation  and  maintenance  funds.  The  fund's  accounting  practices  and 
related  requirements  for  disclosure  of  information  are  intended  to  help  managers  and 
workers  focus  on  production  costs  and  performance.  However,  the  Department  of  Defense 
(DOD)  has  long-standing  problems  in  accumulating  and  reporting  the  full  costs  associated 
with  its  working  capital  fund  operations. 

^  Direct  appropriations  are  sometimes  referred  to  as  mission  funding  by  DOD  and  Navy 
officials.  They  are  funds  appropriated  direcdy  by  the  Congress  that  authorize  agencies  to 
incur  obligations  for  designated  purposes.  An  agency  may  not  spend  more  than  the  amount 
appropriated. 
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Furthermore,  in  recent  years  there  has  been  increasing  Navy 
dissatisfaction  with  the  costs  associated  with  the  working  capital  fund. 
Given  the  nature  of  the  fund  and  its  emphasis  on  fully  recovering  costs  of 
the  services  provided,  the  perception  sometimes  exists  that  the  fund 
included  fees  and  charges  that  inflated  ship  maintenance  costs  compared 
to  mission  funding,  which  does  not  reflect  the  full  cost  of  operations.  In 
accordance  with  one  of  the  basic  tenets  of  a  working  capital  fund,  the  costs 
associated  with  ship  maintenance  are  more  visible  when  financed  using 
this  mechanism.  While  the  level  of  resources  required  to  carry  out  ship 
maintenance  activities  is  likely  to  be  similar  regardless  of  whether  financed 
using  mission  funding  or  through  a  working  capital  fund,  under  the 
working  capital  fund  the  customer  is  more  likely  to  be  directly  responsible 
for  a  larger  portion  of  those  costs.  Use  of  a  working  capital  fund  better 
enables  DOD  components  to  fully  account  for  their  share  of  the  program 
costs.  In  commenting  on  a  draft  of  this  report,  the  Navy  said  that  it  chose 
the  Shipyard  Management  Information  System  as  the  cost  accounting 
system  for  the  Pearl  Harbor  Pilot.  According  to  the  Navy,  this  system 
(which  is  used  by  other  naval  shipyards  under  the  NWCF)  provides  the 
pilot  the  same  degree  of  cost  visibility  as  a  working  capital  fund  activity. 

Full  accounting  for  the  costs  of  federal  programs  is  required  by  federal 
accounting  standards.  Specifically,  Statement  of  Federal  Financial 
Accounting  Standards  No.  4  requires  federal  agencies  to  accumulate  the 
full  cost  of  outputs  through  appropriate  costing  methodologies  or  cost 
finding  techniques  (the  full  cost  of  an  output  is  the  sum  of  (1)  the  cost  of 
resources  consumed  that  directly  or  indirectly  contributes  to  the  output 
and  (2)  the  cost  of  identifiable  supporting  services  provided  by  other 
entities). 


After  Expressing  Concerns, 
Department  of  Defense 
Officials  Approved  the  Test 
Plan  in  September  1998 


In  1998,  as  Navy  officials  were  formulating  plans  for  the  Pearl  Harbor  Pilot, 
DOD  officials  expressed  concern  that  the  Navy's  test  plan  and  baseline  data 
would  not  adequately  measure  the  pilot's  impact  on  productivity  and 
performance.  Consequently,  the  Navy  revised  the  test  plan  and  tried  to 
minimize  the  data  limitations  in  the  baseline.  Ultimately,  as  shown  in  table 
1,  the  Navy  selected  nine  test  metrics  to  evaluate  the  pilot's  effectiveness. 
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Table  1:  The  Navy's  Nine  Test  Metrics  for  Evaluating  the  Peari  Harbor  Pilot's 
Effectiveness 


Metric 

1 .  Total  cost  of  a  production  shop,  direct  labor  hour  of  work  delivered  to  the  customer 

(indicator  of  the  pilot's  efficiency  in  terms  of  the  cost  per  direct  maintenance  hour) _ 

2.  Total  labor  hours  expended  to  deliver  a  production  shop,  direct  labor  hour  to  the 

customer  (indicator  of  the  pilot's  productivity  in  terms  of  personnel  utilization) _ 

3.  Total  Current  Ship  Maintenance  Program  work  items  completed  (indicator  of  the  pilot’s 

productivity  in  terms  of  the  number  of  work  items  completed) _ 

4.  Total  Current  Ship  Maintenance  Program  work  items  in  the  backlog  (indicator  of  the 

material  conditions  of  PACFLT  ships  In  terms  of  work  items  not  yet  completed) _ 

5.  Schedule  adherence  of  CNO  maintenance  projects  (indicator  of  customer  satisfaction 

in  terms  of  completing  projects  on  schedule) _ 

6.  Rework  index  for  CNO  maintenance  projects  (indicator  of  the  pilot's  quality  of  work  in 

terms  of  hours  required  to  correct  work  deficiencies) _ 

7.  Activity  work  schedule  integrity  index  (indicator  of  customer  satisfaction  in  terms  of 

completing  projects  on  schedule) _ 

8.  Casualty  reports  caused  by  activity  work  (indicator  of  the  pilot's  quality  of  work  in  terms 

of  casualty  reports — ship  reports  indicating  equipment  failure) _ 

9.  Earned  value  (indicator  of  the  pilot's  productivity  in  terms  of  hours  to  complete  similar 
work  items) 


Source:  The  Pearl  Harbor  Pilot  Test  Plan,  as  amended,  Department  of  the  Navy  (June  30, 1998). 

In  June  1998,  the  DOD  Inspector  General  had  recommended  the  Navy  delay 
the  pilot  and  restart  the  data  collection  process,  applying  the  same  data 
collection  methodologies  and  controls  for  the  baseline  that  would  be  used 
to  collect  data  during  the  pilot  period.  According  to  DOD  Comptroller  and 
Inspector  General  officials,  the  most  severe  of  the  metrics'  limitations 
affected  the  baseline  costs  for  the  first  two  test  metrics  shown  in  table  1. 
For  example,  military  personnel  costs  were  not  traditionally  collected  by 
intermediate  maintenance  facilities  because  these  facilities  were  largely 
funded  out  of  operation  and  maintenance  appropriations  while  military 
personnel  costs  were  already  paid  for  separately  out  of  military  personnel 
appropriations.  Consequently,  military  personnel  costs  were  also 
accounted  for  separately.  In  addition,  some  labor  hours  appeared  to  be 
inaccurate.  One  DOD  contractor  reported  the  NIMF  database  contained 
zero  labor  hours  for  approximately  6  percent  of  the  maintenance  tasks 
completed  by  the  facility.  Further,  some  Navy  officials  expressed  concern 
that  some  recorded  labor  hours  appeared  to  reflect  the  expected  work 
standards  and  not  actual  production  hours. 
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In  response  to  the  DOD  Inspector  General  recommendation,  Navy  officials 
said  the  pilot  was  too  far  along  in  the  integration  process  in  June  1998  to 
reestablish  separate  maintenance  activities  without  incurring  significant 
cost  and  operational  turmoil.  The  expected  cost  and  turmoil  of  such  a 
reversal,  according  to  these  officials,  would  outweigh  any  potential  benefit 
obtained  from  developing  a  better  baseline.  Consequently,  the  Navy  revised 
the  test  plan  to  minimize  the  data  limitations.  For  example,  as  CNO  and 
NAVSEA  officials  requested,  the  Naval  Audit  Service  (NAS)  agreed  to 
validate  the  baseline  and  pilot  results  for  six  of  the  nine  test  metrics.^  In 
several  instances,  baseline  data  were  routinely  collected  by 
preconsolidation  activities,  and  they  are  available  to  provide  a  basis  for 
comparison  with  postconsolidation  performance  data.  In  addition,  as 
requested  by  CNO  and  NAVSEA  officials,  the  NAS  developed  a 
methodology  to  estimate  baseline  values  for  the  first  two  metrics.  The 
methodology  is  based  on  the  Navy's  personnel  wages  and  standards,  as 
well  as,  on  the  best  available  data  from  the  former  NIMF.  Although  the 
resulting  baseline  estimates  for  the  two  metrics  may  not  be  exact,  Navy 
officials  believe  they  are  comparable  to  historical  trends  for  intermediate 
maintenance  facilities  and,  consequently,  provide  a  reasonable  baseline  on 
which  to  indicate  the  pilot's  impact  on  productivity  and  performance.  On 
September  14, 1998,  the  Principal  Deputy  Under  Secretary  of  Defense 
(Comptroller)  approved  the  Navy's  test  plan.  Appendix  II  provides  more 
detail  concerning  the  evolution  of  the  test  plan  and  the  nine  metrics. 


Congressional 
Requirements  for  the  Pearl 
Harbor  Pilot 


The  Conference  Report  for  the  DOD  Appropriations  Act  for  Fiscal  Year 
1998^  recognized  that  the  Navy  planned  to  initiate  a  pilot  to  study  whether 
combining  fleet  intermediate  maintenance  facilities  with  Navy  shipyards 
might  yield  economies  of  scale  and  allow  maintenance  managers  to  better 
balance  workloads.  The  conferees  concluded  that  it  would  take  at  least 
2  years  before  the  Navy  could  determine  whether  this  new  arrangement 
was  effective  and  should  be  made  permanent  or  expanded  to  other 
locations.  Further,  the  report  directed  the  Navy  not  to  expand  the  pilot  until 
6  months  after  it  reported  to  the  Committees  on  Appropriations  on  its 
findings,  and  that  such  report  be  made  on  or  after  April  1,  1999.  The 
conferees  also  directed  the  Navy  not  to  make  any  permanent  changes  to 


^  Previously,  the  NAS  had  agreed  to  collect  and  validate  the  baseline  and  pilot  results  for  the 
first  two  test  metrics. 

'  House  Report  105-265,  September  23,  1997. 
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the  workforce  in  terms  of  total  number  of  employees  or  any  other 
permanent  changes  until  the  pilot  had  been  completed  and  evaluated  by  the 
Congress.  Currently,  the  Navy  plans  to  issue  an  interim  report  on  the  pilot 
shortly  after  October  1999. 


While  the  Pilot  Has 
Potential  to  Improve 
Maintenance  Activities, 
Preliminary  Data  Show 
Mixed  Results 


Preliminary  results  of  the  Pearl  Harbor  Pilot  have  been  mixed,  showing 
either  improvements,  no  improvements,  or  insufficient  data  to  determine 
results.  For  most  of  the  measures  where  data  are  available,  indications  are 
that  the  pilot  has  the  potential  to  improve  maintenance  activities  in  Hawaii. 
While  the  test  plan  calls  for  using  nine  metrics,  the  data  has  been  gathered 
for  only  five  of  them.  For  these  five  metrics,  preliminary  results  for  two 
metrics  indicate  improvements  that  meet  or  exceed  the  Navy's 
expectations:  two  others  indicate  improvements  that  fall  slightly  short  of 
the  Navy's  expectations;  and  one  metric  indicates  no  improvement.  Other 
positive  results  include  the  fact  that  the  Navy  has  integrated  the 
workforces  from  two  work  centers  into  a  common  pool,  increased 
management  flexibility  in  assigning  workers  to  maintenance  projects  based 
on  evolving  priorities,  and  reduced  the  maintenance  infrastructure  at  Pearl 
Harbor.  At  the  same  time,  other  planned  efficiencies  are  yet  to  be  realized. 
For  example,  the  pilot  has  not  resulted  in  the  number  of  overhead  workers 
projected  to  move  to  direct  maintenance  positions  or  in  the  consolidation 
of  all  industrial  plant  equipment.*  According  to  Navy  officials,  these  actions 
would  have  increased  the  number  of  people  working  directly  on 
maintenance  projects  and  the  utilization  of  existing  plant  equipment. 


The  Pilot  Has  the  Potential 
to  Improve  Maintenance 
Activities  in  Hawaii 


The  nine  metrics  chosen  for  the  Pearl  Harbor  Pilot  Test  Plan  represent  a 
variety  of  issues  and  performance  indicators.  Although  data  available  for 
some  of  the  metrics  are  limited,  it  is  sufficient  to  provide  a  general,  if 
imprecise,  indication  of  the  pilot's  impact  on  productivity  and 
performance.  In  this  case,  as  indicated  in  table  2,  while  test  results  have  not 
yet  been  fully  developed,  the  preliminary  results  have  been  mixed,  showing 
either  improvements,  no  improvements,  or  insufficient  data  to  determine 


^  In  general,  overhead  workers  are  those  employees  who  are  not  working  directly  on 
maintenance  projects  and  are  assigned  to  administrative  and  support  positions  such  as 
personnel  and  financial  departments.  Direct  maintenance  workers  are  those  employees  who 
accomplish  work  directly  identifiable  to  a  maintenance  project  such  as  repair,  maintenance, 
and  installation  of  equipment  and  components. 
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results.  Where  data  are  available,  overall  indications  are  that  the  pilot  has 
the  potential  to  improve  maintenance  activities  in  Hawaii. 

Table  2:  Preliminary  Results,  Strengths,  and  Weaknesses  of  the  Pearl  Harbor  Pilot  Test  Metrics 

Metric  (number)  and 
anticipated  goal 

Preliminary  results 

Strengths  (S)  and  weaknesses  (W) 

Improvement  indicated 

Total  cost  of  a  production  shop, 
direct  labor  hour  of  work  delivered 
to  the  customer  (1) 

Indicator  of  the  pilot's  efficiency  in 
terms  of  the  cost  per  direct 
maintenance  hour 

Indicates  improvement  because 
preliminary  data  show  that  It  has 
cost  slightly  less  to  deliver  one 
direct  maintenance  hour  In  fiscal 
year  1 998,  but  falls  short  of  the 
Navy's  expectation 

Variant  of  a  traditional  indicator  of  shipyard  efficiency  (S) 

Shipyard  baseline  data  are  considered  reliable  (S) 

NAS  developed  baseline  estimates  and  plans  to  validate  and 
generate  the  test  data  (S) 

NAS  developed  baseline  estimates  of  the  NIMF  military 
personnel  costs  because  the  facility  did  not  maintain  this  type  of 
data  (W) 

NIMF  baseline  data  on  production  hours  have  limitations,  but 
are  considered  reliable  enough  by  Navy  officials  to  provide  a 
rough,  if  imprecise,  estimate  of  the  baseline  (W) 

Total  labor  hours  expended  to 
deliver  a  production  shop,  direct 
labor  hour  to  the  customer  (2) 
Indicator  of  the  pilot's  productivity 
in  terms  of  personnel  utilization 

Indicates  improvement  because 
preliminary  data  show  that  it  has 
taken  fewer  hours  to  deliver  one 
direct  maintenance  hour  in  fiscal 
year  1998 

Variant  of  a  traditional  indicator  of  shipyard  productivity  (S) 
Shipyard  baseline  data  are  considered  reliable  (S) 

NAS  developed  baseline  estimates  and  plans  to  validate  and 
generate  the  test  data  (S) 

NIMF  baseline  data  on  production  hours  have  limitations  but  are 
considered  reliable  enough  by  Navy  officials  to  provide  a  rough, 
if  Imprecise,  estimate  of  the  baseline  (W) 

Total  Current  Ship  Maintenance 
Program  work  Items  in  the 
backlog  (4) 

Indicator  of  the  material 
conditions  of  PACFLT  ships  in 
terms  of  work  items  not  yet 
completed 

Indicates  Improvement  because 
preliminary  data  show  that  there 
are  fewer  backlogged  work  items 
since  the  consolidation,  but  falls 
short  of  the  Navy's  expectation 

Traditionally  considered  an  indicator  of  the  material  condition  of 
ships  (S) 

Fleet  and  ship  commanders  maintain  baseline  and  test  data, 
which  are  considered  reliable  (S) 

NAS  plans  to  validate  the  baseline  and  test  data  (S) 

Is  an  indicator  of  the  NIMF  and  the  combined  facility's 
productivity  but  not  the  former  shipyard  (W) 

Schedule  adherence  of  CNO  ship 
maintenance  projects  (5) 

Indicator  of  customer  satisfaction 
in  terms  of  completing  projects  on 
schedule 

Indicates  improvement  because 
preliminary  data  show  that  more 
CNO  projects  are  being  completed 
on  time  or  ahead  of  schedule  since 
the  consolidation 

Traditionally  considered  an  indicator  of  the  shipyard  customers' 
satisfaction  (S) 

NAVSEA  maintains  baseline  and  test  data,  which  are 
considered  reliable  (S) 

NAS  plans  to  validate  the  baseline  and  test  data  (S) 

Is  an  indicator  of  shipyard  and  the  combined  facility's 
performance  but  not  the  former  NIMF  (W) 

No  improvement  indicated 

Total  Current  Ship  Maintenance 
Program  work  items  completed 
(3) 

Indicator  of  the  pilot’s  productivity 
in  terms  of  the  number  of  work 
items  completed 

Indicates  no  Improvement  because 
preliminary  data  show  that  there 
are  fewer  work  items  being 
completed  since  the  consolidation 

Fleet  and  ship  commanders  maintain  baseline  and  test  data, 
which  are  considered  reliable  (S) 

NAS  plans  to  validate  the  baseline  and  test  data  (S) 

Is  an  indicator  of  the  NIMF  and  the  combined  facility’s 
productivity,  but  not  the  former  shipyard  (W) 

Is  affected  by  other  factors  not  related  to  the  pilot  (W) 

(continued) 
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Metric  (number)  and 
anticipated  goal 

Preliminary  results 

Strengths  (S)  and  weaknesses  (W) 

Not  yet  determined 

Rework  index  for  CNO 
maintenance  projects  (6) 

Indicator  of  the  pilot's  quality  of 
work  in  terms  of  hours  required  to 
correct  work  deficiencies 

Not  yet  determined  because  the 
Navy  requires  additional  time  to 
develop  the  test  data 

Traditionally  considered  an  indicator  of  the  shipyard  quality  of 
work  (S) 

NAS  plans  to  validate  the  baseline  and  test  data  (S) 

Is  an  indicator  of  shipyard  and  the  combined  facility's 
performance,  but  not  the  former  NIMF  (W) 

Activity  work  schedule  integrity 
index  (7) 

Indicator  of  customer  satisfaction 
in  terms  of  completing  projects  on 
schedule 

Not  yet  determined  because  the 
Navy  does  not  plan  to  gather  data 
needed  to  develop  this  metric 
unless  the  results  of  the  first  six 
metrics  are  inconclusive 

Traditionally  considered  an  indicator  of  the  shipyard  customers' 
satisfaction  (S) 

Shipyard  maintains  baseline  and  test  data,  which  are 
considered  reliable  (S) 

Is  an  indicator  of  shipyard  and  the  combined  facility's 
performance,  but  not  the  former  NIMF  (W) 

Is  similar  to  the  fifth  metric  measuring  schedules  adherence 
and,  as  a  result.  Navy  officials  question  the  usefulness  of 
developing  this  metric  (W) 

Casualty  reports  caused  by 
activity  work  (8) 

Indicator  of  the  pilot's  quality  of 
work  in  terms  of  casualty  reports 

Not  yet  determined  because  the 
Navy  does  not  plan  to  gather  data 
needed  to  develop  this  metric 
unless  the  results  of  the  first  six 
metrics  are  inconclusive 

Fleet  and  ship  commanders  maintain  baseline  and  test  data, 
which  are  considered  reliable  (S) 

Is  an  indicator  of  shipyard  and  the  combined  facility’s 
performance,  but  not  the  former  NIMF  (W) 

Is  similar  to  the  sixth  metric  measuring  rework  and,  as  a  result. 
Navy  officials  question  the  usefulness  of  developing  this  metric 
(W) 

Earned  value  (9) 

Indicator  of  the  pilot’s  productivity 
in  terms  of  hours  to  complete 
similar  work  items 

Not  yet  determined  because  the 
Navy  does  not  plan  to  gather  data 
needed  to  develop  this  metric 
unless  the  results  of  the  first  six 
metrics  are  inconclusive 

Attempts  to  measure  the  pilot's  effect  on  maintenance  outputs 
(S) 

Should  require  significant  resources  to  develop  results  (W) 

Is  based  on  a  limited  universe  of  work  items  (W) 

Note(s):  Results  as  of  June  22, 1999. 

Appendix  II  provides  more  detail  concerning  the  strengths  and  weaknesses  of  the  nine  metrics. 

Source:  Interviews  with  Navy  officials  and  our  analysis  of  the  Pearl  HarborPilot  Test  Plan  dated 
June  30, 1998;  NAS  capacity  evaluation  reports  for  fiscal  years  1997  and  1998;  and  pilot  status 
reports. 

As  shown  in  table  2,  two  test  metrics  indicate  improvements  that  meet  or 
exceed  the  Navy's  expectations  and  two  indicate  improvements  that  fall 
slightly  short  of  the  Navy's  expectations:  only  one  metric  indicates  no 
improvement.  Because  of  the  operational  turbulence  experienced  during 
the  consolidation,  we  believe  that  it  is  reasonable  to  expect  that  some 
productivity  and  performance  indicators  might  show  diminished 
improvement  during  the  transition  year.  In  addition,  as  indicated  in  the 
table,  the  Navy  has  not  gathered  performance  data  for  the  remaining  four 
metrics.  The  Navy  is  in  the  process  of  collecting  test  data  on  the  rework 
index  for  CNO  maintenance  projects.  However,  Navy  officials  do  not  plan 
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to  gather  data  needed  to  develop  the  last  three  metrics  unless  the  test 
results  are  inconclusive,  even  though  the  three  metrics  were  added  to  the 
test  plan  at  the  request  of  the  Principle  Deputy  Under  Secretary  of  Defense 
(Comptroller) . 


Other  Efficiencies  Identified 
During  the  Planning  Process 
Have  Been  Achieved 


In  August  1997,  the  Pearl  Harbor  Regional  Maintenance  Pilot  Executive 
Steering  Committee  issued  the  Pilot  Study  Report  for  the  integration  of  the 
PHNSY  and  NIME®  The  purpose  of  the  report  was  to  outline  the  approach 
for  this  integration.  Among  other  improvements,  the  report  envisioned 
increasing  productivity  and  cost-effectiveness  by  integrating  the  workers  of 
former  PHNSY  and  NIMF  into  a  common  workforce  and  reducing  the  ship 
maintenance  infrastructure  in  Hawaii.  As  planned  in  the  Pilot  Study  Report, 
the  Navy  has  integrated  nearly  4,000  workers  from  two  separate  work 
centers  into  a  common  pool,  increased  management  flexibility  in  assigning 
workers  to  maintenance  projects,  and  reduced  the  ship  maintenance 
infrastructure. 


Prior  to  the  pilot,  it  was  difficult  to  shift  work  or  personnel  between 
maintenance  activities  due  to  multiple  financial  structures.  More 
specifically,  the  Navy  stated  that  NWCF  rules  prohibited  assigning  workers 
to  a  job  without  a  funding  document,  which  made  it  difficult  to  shift  work 
between  maintenance  activities.  Accordingly,  when  administrative  and 
financial  requirements  restricted  the  movement  of  workers,  shipyard 
production  personnel  not  working  on  a  specific  maintenance  project  were 
sent  to  the  excess  labor  shop  to  wait  assignment.  During  this  time,  they 
performed  indirect  work  such  as  facility  maintenance  or  grounds-keeping.^ 
Before  the  pilot,  the  number  of  workers  assigned  daily  to  this  shop  ranged 
between  100  to  200.  By  integrating  the  workforce  under  a  single  financial 
and  command  structure,  production  shops  now  have  more  flexibility  in 
assigning  excess  workers  to  other  projects,  including  projects  historically 
completed  by  the  former  intermediate  maintenance  facility.  Consequently, 
the  number  of  workers  assigned  daily  to  the  excess  labor  shop  has  dropped 
to  below  10. 


®  Pilot  Study  Report  for  an  Integration  of  Pearl  Harbor  Naval  Shipyard  and  Naval 
Intermediate  Maintenance  Facility,  Pearl  Harbor,  Pearl  Harbor  Pilot  Executive  Steering 
Committee  (Aug.  26, 1997). 

^  Limited-duty  personnel  with  medical  injuries  or  health  problems  and  personnel  whose 
security  clearance  levels  were  under  review  were  also  assigned  to  the  excess  labor  shop. 
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In  addition,  as  a  result  of  the  consolidation,  the  Navy  has  reduced  the 
maintenance  infrastructure  at  Pearl  Harbor.  For  example,  11  (114,131 
square  feet)  of  the  27  buildings  previously  used  by  the  former  intermediate 
maintenance  facility  were  vacated  and  turned  over  to  Pearl  Harbor  Naval 
Station.®  While  the  consolidated  facility  will  retain  7  of  the  remaining 
16  NIMF  buildings,  it  plans  to  vacate  another  6  (24,907  square  feet)  and  is 
reviewing  the  disposition  of  3  buildings.®  As  of  June  1999,  the  Pearl  Harbor 
Naval  Station  demolished  or  plans  to  demolish  eight  former  NIMF  buildings 
and  was  able  to  demolish  two  of  its  buildings  after  tenants  moved  into 
vacated  NIMF  buildings.  The  demolition  costs  for  all  10  buildings  is 
estimated  at  $1.9  million  with  a  projected  annual  cost  avoidance  of 
$307,000  or  a  payback  period  of  a  little  over  6  years. 


Some  Efficiencies  Have  Yet  The  Pilot  Study  Report  envisioned  increasing  productivity  and 
to  Be  Fullv  Achieved  cost-effectiveness  by  moving  overhead  workers  to  direct  maintenance 

^  positions  and  consolidating  industrial  plant  equipment.  However,  these 

efficiencies  have  not  yet  been  fully  achieved. 

For  example,  the  consolidation  did  not  result  in  the  projected  number  of 
overhead  workers  moving  to  direct  maintenance  work-an  important 
element  in  increasing  the  combined  facility's  productivity.To  increase 
productivity,  one  of  the  stated  goals  of  the  pilot  was  to  increase  the  number 
of  direct  production  workers  relative  to  the  number  of  supervisors  and 
overhead  personnel  without  increasing  costs.  Logicalfy,  increasing  the 
number  of  production  workers  should  result  in  increased  production. To 
accomplish  this  goal,  the  Pilot  Study  Report  proposed  that  95  civilian 
overhead  workers  be  moved  to  positions  in  direct  maintenance  work. 
However,  only  four  workers  moved.  According  to  Pearl  Harbor  NSY  &  IMF 
officials,  while  they  have  tried,  it  has  been  impracticable  for  them  to  move 
civilians  to  such  positions  because  of  (1)  the  time  required  for  overhead 
workers  to  become  skilled,  usually  several  years;  (2)  regulations  restricting 
the  process  for  downgrades;  and  (3)  potentially  negative  action  by  workers 


®  In  general,  the  Pearl  Harbor  Naval  Station  has  ownership  and  is  responsible  for  the 
management  and  maintenance  of  Navy  buildings  in  Hawaii, 

®  According  to  Navy  officials,  no  former  shipyard  buildings  will  be  vacated  as  a  result  of  the 
Pearl  Harbor  Pilot. 
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and  employee  representatives.  In  addition,  several  department  directors 
and  overhead  supervisors  said  they  were  unwilling  to  release  any  personnel 
because  of  the  increased  workload  due  to  changes  in  administrative  and 
financial  systems  since  the  consolidation.  Furthermore,  other  Navy 
officials  said  that  efforts  to  move  supervisors  to  direct  production  resulted 
in  too  few  supervisors,  which  led  to  problems  in  planning  and  coordinating 
work  on  maintenance  projects.  According  to  CNO  and  NAVSEA  officials, 
more  overhead  workers  will  be  moved  to  direct  maintenance  positions  as 
the  pilot  progresses. 

In  addition,  industrial  plant  equipment  has  not  been  completely 
consolidated  at  integrated  production  shops.  The  Pilot  Study  Report 
suggested  the  in-shop  industrial  equipment  be  efficiently  consolidated  and 
redistributed  to  improve  maintenance  operations.  Of  the  271  items  of 
industrial  equipment  at  the  former  intermediate  maintenance  facility, 

114  items  were  relocated,  132  items  were  mothballed,  and  25  items  were 
kept  operational  at  their  original  location.  According  to  officials  formerly 
associated  with  the  Pearl  Harbor  NSY  &  IMF,  the  1 14  related  items  were 
equipment  that  required  little  or  no  removal  and  installation  costs. 
Although  many  of  the  132  mothballed  items  are  in  better  condition  and 
newer  than  the  same  type  of  equipment  located  currently  in  the 
consolidated  shops,  the  funding  required  to  relocate  the  NIMF  equipment 
has  not  been  available.  Most  mothballed  items  are  semipermanently 
attached  heavy  equipment  that  requires  funding  to  remove,  transport,  and 
install  elsewhere.  According  to  Pearl  Harbor  officials,  congressional 
guidance  directing  the  Navy  not  to  make  any  permanent  changes  until  after 
the  pilot  period  has  to  a  lesser  degree  influenced  the  decision  not  to 
relocate  the  mothballed  items.  They  plan  to  request  funding  to  move 
equipment  after  the  pilot  period  is  completed. 


The  Pearl  Harbor  Pilot  Will 
Serve  Only  as  a  General 
Model  for  Similar 
Consolidations 


The  Pearl  Harbor  Pilot  is  likely  to  serve  only  as  a  general  model  for  future 
consolidation  efforts  because  of  the  unique  aspects  of  ship  maintenance 
activities  in  Hawaii,  such  as  the  close  proximity  of  facilities  and  the 
significant  portion  of  fleet-funded  work.  Likewise,  because  of  this 


Our  prior  work  examining  DOD  reform  initiatives  has  noted  the  importance  of  top 
management  commitment  and  sustained  support  for  reform  initiatives,  as  well  as 
overcoming  culture  barriers  and  resistance  to  change.  See  Defense  Reform  Initiatives: 
Organization,  Status,  and  Challenges  (GAO/NSIAD-99-87,  Apr.  21, 1999). 
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uniqueness,  the  pilot  provides  only  a  general  indication  that  future 
consolidations  will  result  in  similar  efficiencies. 

While  there  can  be  clear  benefits  to  regionalization,  available  statistical 
data  suggest  conditions  may  have  been  more  favorable  to  such  efforts  in 
Pearl  Harbor  than  may  exist  at  other  Navy  locations.  For  example,  during 
fiscal  years  1996  through  1998,  the  Navy  reportedly  replaced  698  military 
personnel  at  the  former  NIMF  with  504  experienced  civilian  workers, 
mostly  from  the  shipyard.  Although  civilian  workers  were  periodically 
transferred  between  intermediate  facilities  and  shipyards  at  other 
locations.  Pearl  Harbor  is  the  only  location  where  a  significant  number  of 
workers  has  recently  been  transferred.  Consequently,  according  to  Navy 
officials,  this  large  cadre  of  civilian  workers  with  recent  shipyard 
experience  should  have  expedited  the  integration  of  the  workforce  by 
requiring  less  training  and  fewer  trade  skill  certifications.  This  condition 
does  not  exist  to  such  a  large  extent  at  other  potential  consolidation 
locations,  except  at  the  Puget  Sound  area.  Table  3  compares  some 
additional  differences  between  Pearl  Harbor  and  other  Navy  locations. 


Table  3:  Comparison  of  Ship  Maintenance  Activities  in  the  Pearl  Harbor,  Puget  Sound,  Portsmouth,  and  Norfolk  Areas 


Dollars  in  millions 

Category 

Pearl  Harbor 

Puget  Sound 

Portsmouth 

Norfolk 

Time  required  to  travel  between  the 

•  5  to  1 0  minutes 

•  20  to  30  minutes  to 

•  2  to  3  hours 

•  30  to  45  minutes  to 

intermediate  maintenance  facilities 

the  Trident  Refit  Facility 

to  Groton, 

the  Shore 

and  the  shipyards 

•  2  to  4  hours  to  Everett, 

Connecticut, 

Intermediate 

Washington,  maintenance 

maintenance  facility 

Maintenance 

facility 

Activity 

Reported  shipyard  revenues 

$379.4 

$1,041.8 

$394.1 

$822.1 

Number  of  military  personnel 

•  Shipyard 

43 

47 

53 

64 

•  Intermediate  maintenance  facility 

641 

1,105 

1,134 

2,516 

or  activity 

Number  of  civilian  personnel 

•  Shipyard 

2,742 

8,823 

3,286 

7,127 

•  Intermediate  maintenance  facility 

554 

967 

149 

23 

or  activity 

Note:  Fiscal  year  1998. 

Source:  Our  analysis  of  interviews  and  documents  obtained  from  Navy  officials. 
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As  indicated  in  table  3,  Pearl  Harbor  facilities  are  located  closer  to  each 
other  than  in  other  Navy  locations.  Consequently,  consolidation  of 
activities  separated  by  greater  distances  at  other  Navy  locations  could  be 
more  difficult  to  manage  and,  after  the  consolidation  is  completed,  could 
possibly  reduce  the  combined  facility’s  flexibility  in  the  management  of  the 
workforce  and  workload.  For  example,  given  the  close  proximity  of 
facilities  in  Hawaii,  workers  with  multiple  and  specialized  skills  from  both 
the  former  PHNSY  and  NIMF  can  move  between  ship  maintenance  projects 
and  emergent  work  in  a  few  minutes  compared  to  other  Navy  locations 
where  transportation  time  could  range  up  to  4  hours.  In  addition,  the 
consolidated  production  shops  in  Hawaii  are  able  within  a  few  minutes  to 
pick  up  and  deliver  replacement  parts  for  maintenance  projects  compared 
to  other  locations,  where  delivery  time  could  range  up  to  4  hours  once  the 
shops  are  consolidated.  Consequentfy,  the  longer  transportation  time 
between  facilities  at  other  Navy  locations  could  extend  the  time  frames  for 
maintenance  projects  compared  to  projects  completed  in  Hawaii.  Further, 
as  indicated  in  table  3,  Pearl  Harbor  facilities  differ  in  terms  of  revenues 
and  staffing  than  in  some  of  the  other  areas.  For  example,  except  for  the 
Portsmouth  Naval  Shipyard,  reported  revenues  and  staffing  for  the  other 
two  shipyards  were  more  than  double  the  amount  of  the  PHNSY.  The 
smaller  mission  at  Pearl  Harbor  should  have  made  the  consolidation  easier. 

In  addition  during  fiscal  year  1997,  as  indicated  in  figure  1,  more  of  the 
PHNSY  workload  was  for  the  fleet  compared  with  the  other  three  naval 
shipyards  that  completed  more  work  for  non-fleet  activities.  According  to 
Navy  officials,  a  single  major  customer  should  simplify  the  integration  and 
management  of  ship  maintenance  activities.  They  indicated  that  it  is  much 
easier  to  schedule  and  budget  for  maintenance  projects  on  behalf  of  one 
primary  and  a  few  secondary  customers  compared  to  meeting  the  varying 
requirements  of  several  different  customers  with  equal  workloads. 
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Figure  1 :  Breakdown  of  the  Naval  Shipyards*  Fiscal  Year  1997  Workloads  by  Major  Customer 


60  Percentage  of  total 
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Source:  Our  analysis  of  NAVSEA  data. 
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In  the  future,  Navy  officials  estimated  that  nearly  90  percent  of  the  Pearl 
Harbor  NSY  &  IMF  workload  would  be  for  PACFLT.  The  remaining  10 
percent  will  be  completed  under  a  reimbursable  basis  for  other  activities. 
According  to  Navy  officials,  the  large  share  of  PACFLT  workload  in  Hawaii 
should  reduce  the  need  for  the  combined  maintenance  activity  to  broker 
large  workloads  between  different  Navy  activities.  At  the  same  time,  having 
one  common  financial  system  should  eliminate  the  need  for  potentially 
unwieldy  financial  processes  and  procedures  to  manage  and  budget  for 
different  work  and  funding  sources.  In  other  locations,  NAVSEA  and  the 
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respective  fleet  are  estimated  to  share  the  ship  maintenance  workload  on  a 
more  equitable  basis. 


Issues  Related  to 
Financial  and 
Organizational 
Structures  Need  to  Be 
Resolved 


The  Pearl  Harbor  Pilot  has  sharpened  the  debate  over  the  most  appropriate 
financial  and  organizational  structures  for  such  consolidated  activities. 
Conflicting  views  continue  to  exist  within  the  Departments  of  Defense  and 
the  Navy  over  whether  these  consolidated  activities  should  operate  under 
direct  appropriations,  the  NWCF,  or  a  combination  of  the  two  financial 
structures,  particularly  in  terms  of  maintaining  visibility  over  total 
operational  costs.  Similarly,  little  progress  has  been  made  toward  resolving 
the  departments’  differences  over  the  most  appropriate  organizational 
structure  to  manage  these  types  of  activities.  Consequently,  we  believe 
these  issues  need  to  be  resolved  to  facilitate  effective  and  efficient 
consolidations. 


Issues  Remain  Unresolved 
Regarding  Financial 
Structure 


The  Navy's  proposal  recommending  direct  appropriations  for  the  Pearl 
Harbor  Pilot  stated  that  the  pilot  provided  an  actual  demonstration  of  using 
direct  appropriations  for  a  “major  maintenance  activity,”  which  would  help 
to  determine  the  ultimate  financial  structure  for  the  Navy's  regionalization 
actions.  The  proposal  further  stated  that  a  common  financial  structure 
needed  to  be  tested  by  means  of  the  pilot.  However,  the  Navy's  test  plan 
does  not  attempt  to  measure  the  impact  of  using  a  single  financial  structure 
or  the  benefit  of  using  direct  appropriations  instead  of  the  working  capital 
fund.  At  the  same  time,  DOD  and  Navy  officials  continue  to  have  different 
views  on  the  potential  impact  of  using  direct  appropriations  for  the  pilot  on 
the  productivity  and  performance  of  the  Pearl  Harbor  Pilot  and  future 
regionalization  actions.  The  same  is  true  about  the  command  structure  to 
be  used  for  such  consolidated  operations.  Consequently,  we  believe  these 
issues  need  to  be  resolved  to  facilitate  effective  and  efficient 
consolidations. 


Navy  Selected  Direct  Prior  to  the  consolidation,  the  Navy  recognized  that  the  differences 

Appropriations  between  activity  management  and  financial  structure  inhibited  the 

PHNSY's  and  NIMF's  ability  to  readily  share  workloads  and  resources. 
According  to  Navy  officials,  the  multi-financial  and  command  structures 
required  them  to  use  cumbersome,  workaround  procedures  to  share 
workloads  and  workers  between  the  former  PHNSY  and  NIMF.  Therefore, 
the  shipyard  and  intermediate  maintenance  facility  did  not  readily  share 
workloads  and  resources  because  of  the  time  and  cost  required  to  carry-out 
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Views  on  the  Financial  Structure 
Differ 


the  process.  To  achieve  a  fully  integrated  organization,  the  Navy  decided 
the  Pearl  Harbor  Pilot  should  use  a  single  financial  structure  and  selected 
direct  appropriations  instead  of  the  NWCF.  This  decision  was  based  on 
several  factors,  but  primarily  because  PACFLT  was  the  larger  customer  of 
the  two  organizations  and  direct  appropriations  matched  the  fleet's 
financial  structure.  Consequently,  Navy  officials  expected  fewer  financial 
issues  under  direct  appropriations  than  NWCF  because  PACFLT  could 
integrate  the  pilot  into  its  existing  financial  structure  without  establishing 
another  system. 

DOD  and  Navy  officials  have  different  views  regarding  the  financial  impact 
of  removing  the  PHNSY  from  the  NWCF.  There  are  several  centrally 
managed  functions  in  support  of  all  shipyards  for  which  the  costs  are 
recovered  in  the  shipyard  labor  rates  under  the  working  capital  fund 
structure,  which  are  not  normally  captured  under  direct  appropriations. 
According  to  DOD  officials,  these  functions  include  NAVSEA  personnel 
directly  supporting  shipyards,  the  Navy's  Crane  Center  that  manages  all 
crane  procurements  and  repair  projects,  centralized  automated 
information  systems,  programming  support,  and  other  centrally  managed 
items.  In  effect,  removing  the  PHNSY  from  the  fund  without  addressing  the 
contribution  (approximately  14  percent)  it  made  to  cover  centrally  funded 
costs  for  naval  shipyards  would  increase  their  rates.  However,  CNO  and 
NAVSEA  officials  believe  the  PHNSY's  share  of  the  overhead  cost  was  not 
large  enough  to  have  a  significant  effect  on  the  other  shipyards'  rates. 

As  shown  in  figure  2,  the  former  PHNSY  generated  1.7  percent  of  the  fund's 
$21.9  billion  in  total  revenues  during  fiscal  year  1998  and  the  other 
shipyards  contributed  another  10.6  percent,  or  $2.3  billion. 
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Figure  2:  Comparison  of  Reported  NWCF  Revenues  by  Activity  Group  for  Fiscal 
Year  1998 
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Note:  The  “other  maintenance"  category  Includes  aviation  and  U.S.  Marine  Corps  depot  maintenance 
activities  and  “other"  includes  ordnance,  information  services,  and  transportation  activities. 

Percentages  do  not  add  to  1 00  percent  due  to  rounding. 

Source:  Our  analysis  of  the  DOD  Inspector  General  report  entitled  Inspector  General,  DOD,  Oversight 
of  the  Naval  Audit  Service  Audit  of  the  Navy  Working  Capital  Fund  Financial  Statements  for  FY 1998 
(99-099,  Mar.  1.  1999). 

DOD  officials  are  concerned  that  the  Navy  has  not  adequately  addressed  a 
potentially  more  significant  issue  regarding  the  impact  of  removing  all 
naval  shipyards  from  the  NWCF.  They  believe  the  Navy  decision  to  remove 
the  shipyards  from  the  fund  would  cause  other  Navy  activities  remaining  in 
the  fund  to  absorb  a  larger  share  of  overhead  costs.  However,  CNO  and 
NAVSEA  officials  believe  that  the  shipyards'  share  is  not  large  enough  to 
have  a  significant  effect  on  the  activities  remaining  in  the  fund. 

Further,  DOD  and  Navy  officials  have  different  views  on  other  financial 
issues: 

•  Working  capital  funds  are  not  directly  subject  to  the  annual 

appropriation  cycle  and  can  continue  operations  without  interruption 
between  fiscal  years.  Consequently,  the  NWCF  provided  former 
shipyard  operators  with  significant  financial  flexibility  because  they 
were  freed  from  reprogramming  limitations  and  restrictions  applicable 
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to  regular  appropriations  and  allowed  to  incur  costs  without  waiting  for 
enactment  of  an  appropriation.  Because  this  financial  flexibility  is 
considered  critical  to  shipyard  operations,  DOD  officials  are  concerned 
that  the  Navy  has  not  addressed  the  impact  of  eliminating  this  flexibility 
on  the  pilot  operations  or  future  consolidations. 

•  The  Congress  established  working  capital  funds,  among  other  purposes, 
to  provide  a  more  flexible  funding  mechanism  that  would  allow  DOD 
industrial,  commercial,  and  service  activities  to  operate  on  the  same 
basis  as  the  private  sector,  including  the  use  of  standard  cost  accounting 
practices  and  widely-accepted  management  techniques.  According  to 
DOD  officials,  applying  these  practices  and  techniques  to  naval 
shipyards  has  provided  cost  visibility  and  improved  the  efficiency  of 
operations.  Although  we  found  that  the  Navy  has  retained  the  cost 
visibility  of  the  former  shipyard  under  the  pilot,  the  cost  information  is 
not  routinely  reported  to  the  major  commands  as  it  was  for  the  NWCF. 
Further,  the  visibility  is  maintained  by  continuing  to  operate  the  former 
shipyard  accounting  and  management  system,  which  could  be 
eliminated  after  the  pilot  is  completed.  For  example,  DOD  officials  are 
concerned  that  the  visibility  will  not  be  maintained  after  the  pUot  period 
is  completed.  Navy  officials  said  they  plan  to  maintain  cost  visibility  of 
the  maintenance  operations  in  Hawaii  even  after  the  pilot  period 
because  the  visibility  is  useful  in  the  management  of  those  operations. 
Whichever  financing  method  is  adopted  after  the  completion  of  the 
pilot,  the  Statement  of  Federal  Financial  Accounting  Standards  No.  4 
requires  DOD  and  the  Navy  to  continue  to  measure  and  report  the  full 
cost  of  such  ship  maintenance  operations  by  responsibility  segment. 

•  The  buyer/seller  relationship  is  a  fundamental  aspect  of  working  capital 
funds.  According  to  DOD  officials,  this  relationship  established  dynamic 
interplay  between  the  buyer  and  the  provider  that  encouraged  cost 
control  and  responsiveness  to  customer  requirements.  In  addition,  it 
provided  the  opportunity  for  shipyard  customers  to  focus  their  attention 
on  accomplishing  their  respective  missions  and  shipyard  operators  to  be 
accountable  and  responsive  to  customers'  needs.  Absent 
reimbursements  for  most  maintenance  transactions  at  Pearl  Harbor 
under  direct  appropriations,  CNO  and  NAVSEA  officials  said  that  a 
buyer/seller  relationship  has  largely  been  eliminated.  They  estimated 
that  only  10  percent  of  the  combined  facility's  future  workload,  such  as 
ship  alterations  and  inactivations,  will  continue  to  be  on  a  reimbursable 
basis  after  the  pilot  period. 

•  According  to  DOD  officials,  another  important  aspect  of  working  capital 
funds  is  their  capital  investment  program.  In  the  case  of  the  NWCF,  the 
former  shipyard  depreciated  its  capital  assets  and  collected  this 
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Discussions  of  Other  Financial 
Structures  Continue 


Differing  Views  Remain 
Regarding  Organizational 
Structure 


expense  from  customers  through  its  rates.  Therefore,  the  fund  had  a 
ready  reserve  to  finance  capital  investments  for  the  shipyard.  According 
to  CNO  and  NAVSEA  officials,  NAVSEA  will  budget  for  future  capital 
improvements  for  the  Pearl  Harbor  NSY  &  IMF.  However,  DOD  officials 
are  concerned  that  future  funding  levels  may  be  insufficient  because  of 
uncertainties  in  the  DOD  budgeting  and  appropriation  process. 

Although  other  similar  regionalization  efforts  are  likely,  according  to  Navy 
officials,  they  may  not  look  like  the  Pearl  Harbor  Pilot  with  direct 
appropriations  as  the  single  financial  structure.  For  example,  Navy  officials 
are  currently  discussing  using  either  direct  appropriations  or  the  working 
capital  fund  to  finance  the  combined  activity  of  future  consolidations,  or  a 
combination  of  the  two  financial  structures.  One  combination  being 
discussed  involves  using  direct  appropriations  to  finance  the  overhead  and 
military  personnel  of  the  combined  facility  while  using  the  NWCF  to 
finance  direct  maintenance  work  and  materials.  Because  they  believe  that 
the  working  capital  fund  is  more  costly  and  less  flexible  than  direct 
appropriations,  several  Navy  officials  are  reluctant  to  use  the  fund  to 
finance  the  cost  of  intermediate  maintenance  activities,  which  are 
currently  financed  with  direct  appropriations. 


The  Navy  has  made  little  progress  toward  resolving  questions  over  the 
appropriate  organizational  structure  for  such  consolidations.  While  the 
proposal  recommending  the  pilot's  financial  structure  stated  that  common 
ownership  was  believed  to  be  beneficial  to  the  Navy's  regional 
maintenance  initiative,  it  needed  to  be  tested  by  the  pilot.  However,  the 
Pearl  Harbor  Pilot  Test  Plan  does  not  attempt  to  measure  the  impact  of 
providing  PACFLT  ownership  and  management  responsibilities  for  the  pilot 
and  Navy  officials  are  uncertain  which  command  structure  should  be  used 
for  similar  consolidations  at  other  locations. 

Although  the  pilot  was  expected  to  streamline  the  maintenance  process 
and  eliminate  stovepipe  organizations,  DOD  and  Navy  officials  do  not 
know  the  extent  to  which  the  pilot’s  current  structure  is  meeting  these 
objectives  or  how  they  might  have  been  met  under  other  command 
structures.  Prior  to  the  pilot,  the  NIMF  operated  as  a  PACFLT  activity  while 
the  former  shipyard  operated  as  a  NAVSEA  activity.  Under  the  pilot. 


In  its  comments  on  a  draft  of  this  report,  the  Navy  said  that  NAVSEA  and  the  Fleet 
Commanders  had  agreed  now  to  operate  future  consolidations  under  direct  appropriations. 
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PACFLT  assumed  ownership  and  overall  management  and  financial 
responsibility  for  the  consolidated  activity  while  NAVSEA  continued  to  be 
the  technical  and  operating  authority.  Although  CNO  and  NAVSEA  officials 
viewed  the  duality  of  fleet  ownership  and  NAVSEA  operation  as  not  adding 
another  management  layer,  according  to  several  DOD  and  other  Navy 
officials,  this  appeared  to  add  a  level  of  responsibility  and  oversight  in  lieu 
of  streamlining  the  former  maintenance  structure  and  eliminating 
organizations.  For  example,  several  Pearl  Harbor  NSY  &  IMF  officials  said 
that  the  current  ownership  and  command  structure  has  resulted  in  the 
practice  of  reporting  and  coordinating  simultaneously  with  both 
commands.  In  addition,  there  has  been  some  confusion  on  which  command 
was  ultimately  responsible  for  collecting  test  data  and  analyzing  the  results 
of  the  pilot.  Navy  officials  are  still  undecided  about  the  appropriate 
organizational  structure  for  these  types  of  activities  and  are  considering 
different  structures  for  other  regionalization  efforts. 


Conclusions  Although  the  Pearl  Harbor  Pilot  is  not  complete,  preliminary  results  have 

been  mixed,  showing  either  performance  improvements,  no  improvements, 
or  insufficient  data  to  determine  results.  For  most  of  the  measures  where 
data  are  available,  indications  are  that  the  pilot  has  the  potential  to  improve 
maintenance  activities  in  Hawaii.  However,  because  of  unique  aspects  of 
ship  maintenance  activities  in  Hawaii,  the  pilot  is  likely  to  serve  only  as  a 
general  model  for  future  consolidation  efforts  and  provides  only  a  general 
indication  that  future  consolidations  will  result  in  similar  efficiencies.  At 
the  same  time,  DOD  and  Navy  officials  continue  to  have  different  views  on 
the  potential  impact  of  using  direct  appropriations  to  finance  the  pilot  on 

(1)  the  Navy  shipyards  and  activities  remaining  in  the  working  capital  fund, 

(2)  ship  maintenance  activities  during  periods  without  appropriations, 

(3)  cost  visibility  of  ship  maintenance  activities,  (4)  incentives  inherent 
under  NWCF's  buyer/seller  relationship  for  improving  productivity  and 
performance,  and  (5)  the  capital  investment  program  for  ship  maintenance 
activities.  Other  issues  that  are  still  unresolved  include  determining 
whether  the  pilot's  organization  structure  under  PACFLT  ownership  has 
helped  streamline  the  ship  maintenance  process  and  improve  operations  in 
Hawaii. 


RoCOmmondationS  Navy  proceeds  with  other  similar  consolidations,  we  recommend 

that  the  Secretary  of  Defense  require  the  Secretary  of  the  Navy  to  resolve 
issues  related  to  the  appropriate  mechanism  to  finance  and  manage  these 
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types  of  activities.  Specific  financial  questions  that  need  to  be  resolved 
include  the  impact  of  using  the  direct  appropriations  to  finance  the  pilot 
and  other  potential  regionalization  actions  with  regard  to  (1)  the  Navy 
shipyards  and  activities  remaining  in  the  working  capital  fund,  (2)  ship 
maintenance  activities  during  periods  without  appropriations,  (3)  cost 
visibility  of  ship  maintenance  activities,  (4)  incentives  inherent  under 
NWCF's  buyer/seller  relationship  for  improving  productivity  and 
performance,  and  (5)  the  capital  investment  program  for  ship  maintenance 
activities.  In  addition,  other  questions  that  need  to  be  resolved  include 
determining  whether  the  pilot's  command  structure  under  the  Fleet's 
ownership  has  helped  streamline  the  ship  maintenance  process  and 
improve  operations  in  Hawaii. 


Agency  Comments  and 
Our  Evaluation 


On  August  5,  1999,  we  requested  comments  on  a  draft  of  this  report  from 
the  Secretary  of  Defense.  On  August  31,  1999,  officials  of  the  Office  of  the 
Assistant  Deputy  Under  Secretary  of  Defense  for  Maintenance  Policy, 
Programs,  and  Resources  said  that  they  concurred  with  the  intent  of  our 
recommendations.  However,  DOD  Comptroller  officials  expressed  concern 
that  the  draft  of  this  report  did  not  adequately  address  the  Navy's 
underlying  assertions  and  assumptions  for  the  pilot  or  whether  the  stated 
objectives  of  the  pilot  were  met.  By  recommending  that  the  Navy  take  steps 
to  address  unresolved  issues  related  to  financial  and  organizational 
structures  as  it  proceeds  with  similar  consolidations,  we  believe  our  report 
addresses  the  most  significant  issues  for  such  consolidations.  Because  the 
pilot  is  not  yet  complete,  we  did  not  attempt  to  determine  whether  the 
stated  objectives  were  fully  met  but,  instead,  reported  the  preliminary 
results  of  the  pilot  on  improving  performance  and  maintenance  activities  in 
Hawaii.  We  believe  that  it  could  take  several  years  for  the  full  effects  of  the 
pilot  to  develop  and,  as  requested,  we  are  reporting  the  preliminary  results. 


In  addition.  Navy  officials  said  that  they  generally  concurred  with  the  draft 
of  this  report  except  for  our  use  of  fiscal  year  1998  data  to  indicate  the 
preliminary  results  of  the  Pearl  Harbor  Pilot,  and  they  advised  that  fiscal 
year  1999  data  would  provide  the  most  valid  assessment  of  the  pilot.  They 
considered  fiscal  year  1998  as  a  transition  year  when  the  maintenance 
activities,  financial  systems,  and  workforce  were  not  fully  integrated  and, 
consequently,  did  not  indicate  the  pilot's  full  performance.  As  discussed  in 
this  report,  we  also  considered  fiscal  year  1998  as  a  transition  year  for  the 
pilot  and  reported  that  the  pilot  was  unlikely  to  result  in  significant 
improvements  during  the  transition  year  because  of  the  operational 
turbulence  experienced  during  the  consolidation.  However,  we  used 
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available  data  from  both  the  transition  year  and  fiscal  year  1999  to  provide 
the  preliminary  results  of  the  pilot.  We  continue  to  believe  that,  in  either 
year,  overall  indications  are  that  the  pilot  has  the  potential  to  improve 
maintenance  activities  in  Hawaii. 

DOD  and  the  Navy  also  provided  technical  corrections  and  clarifications 
and,  where  appropriate,  we  incorporated  them  in  the  report. 


We  are  sending  copies  of  this  report  to  Senator  Pete  V.  Domenici,  Senator 
Daniel  K.  Inouye,  Senator  Frank  R.  Lautenberg,  Senator  Carl  Levin,  Senator 
Joseph  I.  Lieberman,  Senator  Fred  Thompson,  Senator  Ted  Stevens, 
Senator  Charles  S.  Robb,  and  Senator  John  W.  Warner,  and  to 
Representative  John  R.  Kasich,  Representative  Jerry  Lewis,  Representative 
John  P.  Murtha,  Representative  Ike  Skelton,  Representative  Floyd  Spence, 
and  Representative  John  M.  Spratt,  Jr.,  in  their  capacities  as  Chair  or 
Ranking  Minority  Member  of  cognizant  Senate  and  House  Committees  and 
Subcommittees.  We  are  also  sending  copies  of  this  report  to  the  Honorable 
William  S.  Cohen,  Secretary  of  Defense;  the  Honorable  William  J.  Lynn, 
Under  Secretary  of  Defense  (Comptroller) ;  the  Honorable  Richard  Danzig, 
Secretary  of  the  Navy;  and  the  Honorable  Jacob  Lew,  Director,  Office  of 
Management  and  Budget.  Copies  will  also  be  made  available  to  others  upon 
request. 

GAO  contacts  and  key  contributors  to  this  report  are  listed  in  appendix  III. 
Sincerely  yours. 


David  R.  Warren,  Director 
Defense  Management  Issues 
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Scope  and  Methodology 


During  our  review,  we  obtained  data  and  interviewed  Departments  of 
Defense  (DOD)  and  Navy  officials,  including  those  from  the  offices  of  the 
Under  Secretary  of  Defense  (Comptroller) ,  Deputy  Under  Secretary  of 
Defense  for  Logistics,  DOD  Inspector  General,  Assistant  Secretary  of  the 
Navy  for  Financial  Management  and  Comptroller,  Chief  of  Naval 
Operations  (CNO),  Naval  Sea  Systems  Command  (NAVSEA),  the  Pacific 
Fleet  (PACFLT),  Pearl  Harbor  Naval  Shipyard  &  Intermediate  Maintenance 
Facility  (NSY  &  IMF),  San  Diego  Supervisor  of  Ship  Building  Conversion 
and  Repair  Pearl  Harbor  Detachment,  and  the  Naval  Audit  Service  (NAS). 
In  addition,  we  interviewed  officials  of  the  Hawaii  Federal  Employees 
Metal  Trades  Council,  the  Ship  Repair  Association  of  Hawaii,  and 
contractors  involved  in  the  management  and  assessment  of  the  Pearl 
Harbor  Pilot. 

To  identify  the  preliminary  results  of  the  Pearl  Harbor  Pilot,  we  obtained 
and  reviewed  status  reports  and  briefings,  financial  and  human  resources 
documents,  infrastructure  data,  budget  documents,  NAS  capacity 
evaluation  reports  and  documents,  the  Pearl  Harbor  Pilot  Test  Plan  and 
related  guidance,  and  PACFLT  regional  maintenance  and  business  plans.  In 
addition,  we  interviewed  DOD  and  Navy  officials  to  discuss  the  preliminary 
results  of  the  pilot  and  challenges  that  must  be  resolved  to  obtain  the  full 
benefits  of  the  consolidation.  We  observed  the  operations  of  the  Pearl 
Harbor  NSY  &  IMF  and  interviewed  superintendents,  first-line  supervisors, 
and  workers  to  identify  operational  conditions  and  determine  the  pilot’s 
impacts  on  their  maintenance  shops  and  activities.  We  reviewed  the  Pearl 
Harbor  Pilot  Study  Report  and  other  planning  documents  to  identify 
maintenance  procedures  and  processes  expected  to  improve  under  the 
pilot  and  compared  these  expectations  with  actual  operational  results  for 
fiscal  year  1998  and  the  beginning  of  fiscal  year  1999.  When  these  results 
did  not  meet  expectations,  we  discussed  the  differences  with  Navy  officials 
to  determine  the  causes  and  identify  expectations  for  future  operations.  In 
addition,  we  interviewed  officials  of  various  PACFLT  commands 
(customers  of  the  Pearl  Harbor  NSY  &  IMF)  to  determine  whether  the 
combined  facility's  maintenance  activities  and  services  were  comparable  to 
those  provided  prior  to  the  consolidation.  Additionally,  we  interviewed 
Navy  officials  and  collected  data  on  the  apprentice  program,  safety  and 
quality  assurance  programs,  and  issues  related  to  the  morale  of  Pearl 
Harbor  NSY  &  IMF. 

To  assess  the  adequacy  of  the  Pearl  Harbor  Pilot  Test  Plan,  we  reviewed  the 
DOD  Inspector  General's  preliminary  assessment  of  the  test  plan  and 
baseline  data.  We  also  interviewed  DOD,  Navy,  and  contractor  officials 
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about  the  methodology  for  the  test  plan,  reliability  of  the  data,  and 
strengths  and  weaknesses  of  each  test  metric.  To  obtain  an  understanding 
of  the  methodology  and  data  reliability,  we  reviewed  the  process  and  data 
used  to  develop  the  fiscal  year  1997  baseline,  the  fiscal  year  1998  transition 
results,  and  the  fiscal  year  1999  preliminary  results  and,  on  a  selected  basis, 
we  traced  the  data  to  their  original  source  documents.  To  identify  potential 
areas  for  improving  the  test  plan,  we  discussed  the  strengths  and 
weaknesses  of  the  test  plan  with  DOD,  Navy,  and  contractor  officials.  We 
also  discussed  several  key  elements  of  the  pilot  that  were  omitted  from  the 
test  plan,  including  the  reasons  for  omission  or  the  potential  need  for 
inclusion. 

To  identify  issues  that  remain  for  future  consolidations  at  other  Navy 
locations,  we  obtained  and  reviewed  status  reports  and  briefings,  financial 
and  human  resources  documents,  the  Pearl  Harbor  Pilot  Test  Plan  and 
related  guidance,  and  PACFLT  regional  maintenance  and  business  plans.  In 
addition,  we  interviewed  DOD  and  Navy  officials  to  discuss  issues  that 
must  be  resolved  to  obtain  the  full  benefits  of  the  Pearl  Harbor 
consolidation  and  other  similar  consolidations  at  other  Navy  locations.  In 
addition,  during  meetings  with  DOD  and  Navy  officials,  we  discussed  the 
advantages  and  disadvantages  of  whether  future  consolidations  should  use 
direct  appropriations,  the  Naval  Working  Capital  Fund  (NWCF),  or  a 
combination  of  the  two  financial  systems.  To  help  assess  the  potential 
impact  of  removing  the  former  Pearl  Harbor  Naval  Shipyard  (PHNSY)  and 
other  naval  shipyards  from  the  NWCF,  we  identified  and  analyzed  the 
percentage  of  revenues  the  former  PHNSY  and  the  other  shipyards 
contributed  to  the  fund  during  fiscal  years  1997  and  1998.  We  also 
discussed  the  potential  impact  of  their  removal  from  the  fund  with  DOD 
and  Navy  officials.  Additionally,  during  meetings  with  DOD  and  Navy 
officials,  we  discussed  the  advantages  and  disadvantages  of  whether 
NAVSEA  or  the  fleets  should  be  responsible  for  the  ownership  and 
management  of  future  consolidations.  To  determine  whether  the  pilot 
could  serve  as  a  model  for  similar  consolidations  at  other  locations,  we 
identified  and  analyzed  reasons  why  the  Navy  selected  Pearl  Harbor 
maintenance  activities  for  the  pilot.  We  compared  these  conditions  with 
those  found  at  the  Puget  Sound,  Portsmouth,  and  Norfolk  areas  and 
discussed  the  potential  impact  of  these  differences  on  other  consolidations 
with  DOD  and  Navy  officials. 

Furthermore,  we  documented  the  reasons  the  Navy  selected  direct 
appropriations,  and  the  pilot's  experience  of  operating  under  direct 
appropriations.  We  also  interviewed  DOD  and  Navy  officials  to  discuss  the 
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actual  level  of  cost  visibility  retained  under  the  pilot  and  the  various  levels 
considered  adequate  for  ship  maintenance  activities.  In  addition,  we 
determined  the  level  of  cost  visibility  retained  for  four  ongoing 
maintenance  projects  for  the  U.S.S.  Charlotte,  U.S.S.  Russell,  U.S.S. 

Tucson,  and  U.S.S.  Crommelin.  Specifically,  we  reviewed  the  start  date, 
estimated  date  of  completion,  estimated  cost,  direct  labor  hours  of  military 
and  civilian  personnel,  overtime  hours,  material  and  supply  cost,  and 
contract  services  for  the  projects.  In  addition,  to  determine  whether  the 
same  level  of  detail  was  being  maintained  by  the  Pearl  Harbor  NSY  &  IMF 
under  direct  appropriations,  we  reviewed  material  expenditures  and  labor- 
hours  for  military  and  civilian  personnel  by  job  order  number  for  the 
projects. 

In  performing  this  review,  we  used  the  same  budget  and  accounting 
systems,  reports,  and  statistics  DOD  and  the  Navy  use  to  manage  and 
monitor  their  ship  maintenance  program.  We  did  not  independently 
determine  the  reliability  of  the  reported  financial  information.  However, 
our  recent  audit  of  the  federal  government’s  financial  statements,  including 
DOD’s  and  the  Navy’s  statements,  questioned  the  reliability  of  reported 
financial  information  because  not  all  obligations  and  expenditures  are 
recorded  to  specific  budgetary  accounts. 

We  conducted  our  review  from  May  1998  to  July  1999  in  accordance  with 
generally  accepted  government  auditing  standards. 
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The  Navy's  test  plan  for  assessing  the  effectiveness  of  the  Pearl  Harbor 
Pilot  evolved  between  December  1997,  when  the  Deputy  Secretary  of 
Defense  first  required  a  test  plan,  and  September  1998,  when  the  last  three 
test  metrics  were  added  to  the  plan.  During  this  evolution,  DOD,  Navy,  and 
audit  agency  officials  expressed  various  concerns  with  the  scope  and 
validity  of  the  test  plan.  Ultimately,  the  Navy  selected  nine  test  metrics  to 
evaluate  the  productivity  and  performance  of  the  pilot. 


Metrics  to  Be  Used  to 
Evaluate  the  Pilot 


After  several  revisions  and  additions,  the  current  test  plan  contains  nine 
metrics  that  capture  unit  cost  per  output,  production  efficiency,  material 
readiness  of  ships  home  ported  at  Pearl  Harbor,  and  customer  satisfaction. 
During  the  initial  planning  period  in  late  1997,  Navy  officials  discussed 
several  metrics  to  measure  the  impact  of  the  pilot  and  initially  agreed  upon 
the  two  shown  in  table  4. 


Table  4:  Navy's  Initial  Two  Metrics  to  Test  Whether  the  Consolidated  Pearl  Harbor  NSY  &  IMF  Increased  Manpower  Utilization 
and  Lowered  Unit  Cost  of  Production 

Metric 

Data  elements 

Calculation 

1 .  Total  cost  of  a  production  shop,  direct 
labor  hour  of  work  delivered  to  the 
customer 

•  Total  costs  of  the  ship  maintenance  activity 

•  Total  production  shop,  direct  labor  hours  delivered 

•  Total  costs  of  the  ship  maintenance 
activity  or  activities  divided  by  the 
total  production  shop,  direct  labor 
hours  delivered 

2.  Total  available  labor  hours  expended 
to  deliver  a  production  shop,  direct  labor 
hour  to  the  customer 

•  Total  available  labor  hours,  including  direct  and 
indirect  hours,  expended  by  the  ship  maintenance 
activity 

•  Total  production  shop,  direct  labor  hours  delivered 

•  Total  available  labor  hours  divided  by 
the  total  production  shop,  direct  labor 
hours  delivered 

Source:  Interviews  with  Navy  officials  and  our  analysis  of  the  Pearl  Harbor  Pilot  Test  Plan  dated 
June  30, 1998. 

In  December  1997,  recognizing  that  the  Navy  had  not  developed  a  plan  or 
established  a  baseline  for  measuring  success  or  failure  for  the  pilot,  the 
Deputy  Secretary  of  Defense  required  the  Navy  to  develop  a  test  plan  to 
measure  the  benefits  of  the  pilot.^  In  addition,  the  Deputy  Secretary  of 
Defense  suggested  that  “productive  man-days  delivered  per  dollar  outlay” 
and  “number  of  civilian  end  strengths  and  military  on-board  it  took  to 


^  Program  Budget  Decision  Number  404,  Deputy  Secretary  of  Defense  (Dec.  11, 1997). 
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produce  the  productive  man-days”  should  be  used  as  key  assessment 
metrics.  Subsequently,  the  Chief  of  Naval  Operations  (CNO)  and  the  Naval 
Sea  Systems  Command  (NAVSEA)  requested  the  Naval  Audit  Service 
(NAS)  determine  the  combined  cost  of  operations  for  the  former  Pearl 
Harbor  Naval  Shipyard  (PHNSY)  and  the  Naval  Intermediate  Maintenance 
Facility  (NIMF)  for  fiscal  year  1997.^  In  an  April  1998  report,  the  NAS 
provided  a  basis  to  compare  the  cost  effectiveness  of  combining  the  two 
maintenance  activities  and  metric  calculations  for  the  two  assessment 
metrics  defined  at  that  time,  which  are  the  first  two  metrics  of  the  current 
test  plan.^  In  the  report,  the  NAS  recognized  that  the  consolidation  was  an 
evolving  process  and  revisions  to  its  report  may  be  necessary  to  reflect 
changes  in  consolidation  decisions.  In  May  1999,  after  making  changes  to 
the  baseline  data  and  fine-tuning  its  methodology,  the  NAS  revised  its 
baseline  cost  and  productivity  estimates.^  The  revised  estimates  did  not 
change  significantly  from  the  initial  estimates. 

In  April  1998,  the  Navy  issued  a  draft  of  the  Pearl  Harbor  Pilot  Test  Plan 
that  included  five  metrics  to  capture  cost  per  unit  output,  production 
efficiency  and  resource  utilization,  material  readiness  of  PACFLT  ships, 
customer  satisfaction,  and  quality.  However,  in  June  1998,  the  DOD 
Assistant  Inspector  General  for  Auditing  stated  that  the  draft  test  plan 
would  not  provide  sufficient  or  supportable  data  to  draw  conclusions  about 
the  effectiveness  of  consolidating  the  operations  of  the  former  shipyard 
and  intermediate  maintenance  facility.  Consequently  in  June  1998,  a  joint 
DOD  and  Navy  team  issued  a  revised  Pearl  Harbor  Pilot  Test  Plan  that 
changed  the  basis  for  the  quality  metric  from  formal  customer  surveys  to 
an  analysis  of  rework  items  and  added  a  metric  measuring  the  number  of 


^  The  Navy  selected  fiscal  year  1997  as  the  baseline  year  because  this  was  the  last  full  year 
the  former  PHNSY  and  NIMF  operated  as  independent  activities.  Fiscal  year  1998  was 
eliminated  because  of  the  operational  turbulence  expected  by  the  consolidation  of  activities 
during  the  year.  Fiscal  year  1996  was  eliminated  because  there  were  separate  surface  ship 
and  submarine  intermediate  maintenance  facilities  operating  during  the  fiscal  year. 

^  Capacity  Evaluation  Report:  Baseiine  Costs  of  Operating  Naval  Intermediate  Maintenance 
Facility  Pearl  Harbor  and  Pearl  Harbor  Naval  Shipyard,  Auditor  General  of  the  Navy  (98- 
0400,  Apr.  1998). 

^  Capacity  Evaluation  Report:  Revised  Baseline  Costs  of  Operations  and  Metrics  for  Naval 
Intermediate  Maintenance  Facility,  Pearl  Harbor  and  Pearl  Harbor  Naval  Shipyard 
Consolidation,  Auditor  General  of  the  Navy  (99-0408,  May  1999). 
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work-items  completed.®  Consequently,  the  Navy  included  four  metrics  in 
addition  to  those  two  metrics  listed  in  the  previous  table  to  its  final  test 
plan.  (See  table  5.) 

Table  5:  Metrics  Three  Through  Six  to  Test  Whether  the  Consoiidated  Pearl  Harbor  NSY  &  IMF  Increased  Manpower  Utilization 
and  Lowered  Unit  Cost  of  Production 

Metric 

Data  eiements 

Calculation 

3.  Total  Current  Ship  Maintenance 
Program  (CSMP)  work  items  completed 

•  Total  number  of  CSMP  work  items  completed  on 
ships  home  ported  at  Pearl  Harbor  during  the  fiscal 
year 

•  None  required,  absolute  number  of 
completed  CSMP  work  items  used 

4.  Total  Current  Ship  Maintenance 
Program  work  items  in  the  backlog 

•  Total  number  of  CSMP  work  items  in  the  backlog  for 
ships  home  ported  at  Pearl  Harbor 

•  None  required,  absolute  number  of 
CSMP  work  items  in  the  backlog  used 

5.  Schedule  adherence  of  CNO 
maintenance  projects 

•  Sum  of  the  differences  in  the  actual  and  scheduled 
completion  dates  for  each  CNO  ship  maintenance 
project  completed  during  the  fiscal  year 

•  Sum  of  the  scheduled  duration  (number  of  days)  for 
each  CNO  maintenance  project  completed  during 
the  fiscal  year 

•  Sum  of  the  differences  in  the  actual 
and  scheduled  completion  dates 
divided  by  the  total  scheduled 
duration  (number  of  days)  for  each 

CNO  scheduled  ship  maintenance 
project  completed  during  the  fiscal 
year 

6.  Rework  index  for  CNO  maintenance 
projects 

•  Sum  of  the  labor  hours  expended  to  correct  work 
deficiencies  for  each  CNO  scheduled  ship 
maintenance  project  completed  during  the  fiscal 
year 

•  Total  production  shop,  direct  labor  hours  delivered 
for  each  CNO  scheduled  ship  maintenance  project 
completed  during  the  fiscal  year 

•  Sum  of  the  labor  hours  expended  to 
correct  work  deficiencies  divided  by 
the  total  number  of  direct  labor  hours 
delivered  for  each  CNO  scheduled 
ship  maintenance  project  completed 
during  the  fiscal  year 

Source:  Interviews  with  Navy  officials  and  our  analysis  of  the  Pearl  HarborPilot  Test  Plan  dated 
June  30, 1998. 

In  September  1998,  the  Principal  Deputy  Under  Secretary  of  Defense 
(Comptroller)  suggested  the  Navy  incorporate  three  additional  test  metrics 
developed  under  contract  by  the  Anteon  Corporation  into  the  revised  June 
1998  test  plan.®  (See  table  6.)  In  response,  the  Acting  Deputy  Chief  of  Naval 


®  The  Pearl  Harbor  NSY  &  IMF  has  developed  three  additional  internal  metrics: 
production-shop  hours  versus  total  available  hours,  direct  production  worker  versus  non¬ 
production  worker,  and  civilian  labor  cost  per  labor  hour.  While  they  are  not  formally  part  of 
the  test  plan,  Pearl  Harbor  NSY  &  IMF  officials  believe  the  metrics  will  assist  them  in 
assessing  the  effectiveness  of  the  pilot. 

®  The  Pearl  Harbor  Pilot  Study,  Anteon  Corporation  (Sept.  10, 1998). 
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Operations  (Logistics)  attached  them  in  an  appendix  E  to  the  plan.  At  this 
time.  Principal  Deputy  Under  Secretary  of  Defense  approved  the  test  plan. 

Table  6:  Metrics  Seven  Through  Nine  to  Test  Whether  the  Consolidated  Pearl  Harbor  NSY  &  IMF  Increased  Manpower  Utilization 
and  Lowered  Unit  Cost  of  Production 

Metric 

Data  elements 

Calculation 

7.  Activity  work  schedule 
integrity  index 

•  Budgeted  quantity  of  work  scheduled  (labor  days) 

•  Actual  quantity  of  work  performed  (labor  days) 

•  Budgeted  allowance  of  work  scheduled  (labor 
days)  divided  by  the  actual  amount  of  work 
performed  (labor  days) 

8.  Casualty  reports®  caused  by 
activity  work 

•  Casualty  reports 

•  Analysis  of  reports  occurring  within  6  months 
following  the  completion  of  maintenance 

9.  Earned  value 

•  Actual  quantity  of  work  performed  (labor  days)  for 
selected  Ship  Work  Line  Item  Numbers  for  the  fiscal 
year 

•  Statistical  analysis  of  actual  quantity  of  work 
performed  (labor  days)  for  selected  Ship  Work 
Line  Item  Numbers  for  fiscal  years  1997  and 

1999 

^Reports  filed  by  the  ship  of  equipment  failures  that  may  indicate  faulty  work  performed  during  more 
recent  maintenance  projects. 

Source:  Interviews  with  Navy  and  Anteon  Corporation  officials  and  our  analysis  of  the  Pearl  Harbor 
Pilot  Test  Plan  dated  June  30,  1998. 

In  November  1998,  the  Director,  Industrial  Capability,  Maintenance  Policy 
and  Acquisition  Logistics  Division,  CNO,  requested  the  NAS  to  assume  the 
overall  lead  and  control  in  the  test  plan  assessment  process.  The  Director 
requested  that  NAS  involvement  be  expanded  to  include  data  collection  for 
all  of  the  test  plan  metrics,  metric  calculation  and  assessment,  and  pilot 
metric  assessment  reporting.  In  response,  the  auditors  agreed  to  continue 
to  develop  and  report  data  for  the  first  two  metrics  in  the  test  plan  and 
verify  the  data  for  metrics  number  3  through  6.  Additionally,  NAS 
considered  the  responsibility  for  data  collection  and  reporting  the  results 
of  the  pilot  to  be  a  management  function.  It  did  not  assume  responsibility 
for  collecting  and  validating  the  data  for  the  contractor-generated  metrics. 


Baseline,  Performance 
Expectations,  and 
Preliminary  Values  for 
the  Pearl  Harbor  Pilot 


Given  the  time  and  data  constraints  in  developing  the  Peart  Harbor  Pilot 
Test  Plan,  the  test  plan  and  assessment  metrics  provide  the  Navy  a 
mechanism  for  evaluating  the  pilot's  productivity  and  performance.  As 
shown  in  table  7,  the  Navy  has  developed  the  values  for  the  fiscal  year  1997 
baseline  and  performance  expectations  for  six  of  the  nine  metrics  and 
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preliminary  values  for  five  metrics.  However,  the  details  for  collecting  and 
analyzing  the  contractor-generated  metrics  have  not  yet  been  determined. 

Table  7:  Baseline,  Performance  Expectations,  and  Preiiminary  Resuits  of  the  Test  Pian 

Metric 

Fiscal  year  1997  baseline 

Expectation 

Preliminary  result 

1 .  Total  cost  of  a  production  shop, 
direct  labor  hour  of  work  delivered 
to  the  customer^ 

•  $135.36  in  current  year 
dollars 

•  $138.13  in  adjusted  dollars'" 

•  3-5  percent  reduction 

•  $135.59  per  hour  in  fiscal  year  1998 

2.  Total  available  labor  hours 
expended  to  deliver  a  production 
shop,  direct  labor  hour  to  the 
customer® 

•  3.15  hours 

•  3-5  percent  reduction 

•  3.04  hours  in  fiscal  year  1998 

3.  Total  Current  Ship  Maintenance 
Program  work  items  completed'" 

•  19,777 

•  Maintain  at  least  the 
same  completion  rate 

•  14,568  items  in  fiscal  year  1998 

•  7,786  items  as  of  May  1999 

4.  Total  Current  Ship  Maintenance 
Program  work  Items  in  the  backlog*" 

•  17,733 

•  10  percent  reduction 

•  16,462  items  as  of  September  30, 1998 

•  16,538  items  as  of  May  1999 

5.  Schedule  adherence  of  CNO 
maintenance  projectsc 

•  1 1 .4  percent  late 

•  To  improve 

•  18.6  percent  late  in  fiscal  year  1998 

•  3.7  percent  early  in  fiscal  year  1999 
through  May  1999 

6.  Rework  index  for  CNO 
maintenance  projectsc 

•  0.086 

•  No  degradation  of  the 
index 

•  Not  yet  determined 

7.  Activity  work  schedule  integrity 
Index 

•  Not  yet  determined 

•  Unclear 

•  Not  yet  determined 

8.  Casualty  reports  caused  by 
activity  work 

•  Not  yet  determined 

•  Unclear 

•  Not  yet  determined 

9.  Earned  value 

•  Varies  by  work  Item 

•  Unclear 

•  Not  yet  determined 

^Reported  by  the  NAS. 

Adjusted  for  Increases  in  military  and  civilian  pay  rates  due  to  Inflation. 

""Re ported  by  the  Pearl  Harbor  NSY  &  IMF,  but  not  yet  verified  by  the  NAS. 

Source:  Interviews  with  Navy  and  Anteon  Corporation  officials  and  our  analysis  of  the  Pearl  Harbor 
Pilot  Test  Plan  dated  June  30,  1998;  NAS  capacity  evaluation  reports  for  fiscal  years  1997  and  1998; 
and  pilot  status  reports. 

Although  the  Navy  has  established  nine  metrics  that  provide  indicators  of 
productivity  and  performance,  the  metrics  have  data  limitations  and 
weaknesses  that  preclude  their  use  for  precise  measurements. 
Nevertheless,  they  provide  a  basis  for  an  order  of  magnitude  comparison.  A 
discussion  of  the  nine  metrics  follows. 
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Cost  of  a  Production  Shop, 
Direct  Labor  Hour  of  Work 
Delivered 


Measuring  the  pilot's  efficiency  in  terms  of  the  cost  per  direct  maintenance 
hour,  the  Navy's  first  metric  is  determined  by  dividing  the  total  activity  cost 
by  the  number  of  production  shop,  direct  labor  hours  delivered  to  the 
customer/  The  NAS  estimated  that  it  cost  $135.36  to  deliver  one  production 
shop,  direct  labor  hour  in  fiscal  year  1997 — the  baseline  year  for  measuring 
the  pilot's  success.  For  the  pilot  to  show  improvement,  this  index  should 
decrease,  indicating  that  since  the  consolidation,  it  cost  less  to  deliver  a 
direct  maintenance  hour.  The  NAS  estimated  that  the  index  was  $135.59 
during  the  fiscal  year  1998  transition,  indicating  some  improvement  when 
compared  with  the  adjusted  baseline  of  $138.13,  but  falling  short  of  Navy's 
performance  expectation  of  3-  to  5-percent  reduction  in  the  index.®  The 
NAS  adjusted  the  baseline  to  account  for  differences  in  military  and  civilian 
wages  and  inflation  between  fiscal  years  1997  and  1998.  Because  of  the 
operational  turbulence  experienced  during  the  consolidation,  we  believe 
this  indicator  was  unlikely  to  show  a  significant  improvement  during  the 
transition  year. 


Strengths  and  Weaknesses  In  analyzing  this  metric,  we  noted  strengths  and  weaknesses.  The  strengths 

of  this  metric  are  that  the  baseline  is  partly  based  on  shipyard  data,  which 
are  considered  reliable,  and  the  NAS  developed  the  baseline  estimates  and 
plans  to  validate  and  generate  the  test  data.  However,  DOD  and  Navy 
officials  have  expressed  concern  with  the  accuracy  of  the  intermediate 
maintenance  data  collected  during  the  baseline  year,  which  is  considered  a 
weakness.  For  example,  military  personnel  costs  were  not  collected 
because  intermediate  maintenance  facilities  traditionally  did  not  collect 
these  costs.  In  addition,  some  production  hours  appeared  to  be  inaccurate. 
For  example,  a  DOD  contractor  reported  the  former  NIMF  database 
contained  zero  labor  hours  for  approximately  6  percent  of  the  maintenance 
tasks  completed  by  the  facility.  Consequently,  to  strengthen  the  metric,  the 
NAS  developed  a  methodology  that  used  Navy  personnel  wages  and 
standards  and  the  best  available  NIMF  data  to  estimate  a  baseline  for  this 
metric.  Although  the  NIMF  baseline  data  on  military  personnel  costs  and 


^  The  production  shops  consist  of  the  crane,  ship  fitter,  sheet  metal,  welding,  inside 
machine,  outside  machine,  boilermaker,  electrical,  pipe  fitter,  wood/fabric,  cryptographic 
shore  electronics,  electronics,  shipwright/rigging,  and  temporary  services  shops  and  the 
Navy  calibration  and  nuclear  regional  maintenance  departments. 

®  Fiscal  Year  1998  Interim  Costs  for  the  Consolidated  Pearl  Harbor  Naval  Shipyard  and 
Intermediate  Maintenance  Facility,  Pearl  Harbor,  Auditor  General  of  the  Navy  (99-0400, 
June  1,  1999). 
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production  hours  have  limitations,  they  are  considered  reliable  enough  by 
Navy  officials  to  provide  a  rough,  if  imprecise,  estimate  of  the  baseline. 

Labor  Hours  Expended  to 
Deliver  a  Production  Shop, 
Direct  Labor  Hour 

Measuring  the  pilot's  productivity  in  terms  of  personnel  utilization,  the 
second  metric  is  determined  by  dividing  the  total  number  of  avaUable  labor 
hours  by  the  number  of  production  shop,  direct  labor  hours  delivered  to 
the  customers.  The  NAS  estimated  that  it  took  3.15  activity  labor  hours 
(overhead  and  direct  maintenance  hours)  to  deliver  one  production  shop, 
direct  labor  hour  in  fiscal  year  1997.  For  the  pilot  to  show  improvement, 
this  index  should  decrease  indicating  that  it  has  taken  fewer  overhead 
hours  to  generate  1  direct  maintenance  hour  since  the  consolidation.  The 
NAS  reported  that  the  index  decreased  to  3.04  hours  in  fiscal  year  1998, 
indicating  a  3.5-percent  improvement  during  the  transition  year.  This  met 
the  low  end  of  the  Navy's  performance  expectation  of  3-  to  5-percent 
reduction  in  value. 

Strengths  and  Weaknesses 

In  our  analysis,  we  noted  the  strengths  and  weaknesses  of  this  metric.  The 
strengths  of  this  metric  are  that  the  baseline  is  partly  based  on  shipyard 
data,  which  are  considered  reliable,  and  the  NAS  developed  the  baseline 
estimates  and  plans  to  validate  and  generate  the  test  data.  However,  as 
discussed  previously,  DOD  and  Navy  officials  have  expressed  concern  with 
the  intermediate  maintenance  data  collected  during  the  baseline  year. 
Consequently,  to  strengthen  the  metric,  the  NAS  developed  a  methodology 
that  used  best  available  NIMF  data  to  estimate  a  baseline  for  this  metric. 
Although  the  NIMF  baseline  data  on  production  hours  have  weaknesses, 
they  are  considered  reliable  enough  by  Navy  officials  to  provide  a  rough,  if 
imprecise,  estimate  of  the  baseline. 

Current  Ship  Maintenance 
Program  Work  Items 
Completed 

Measuring  the  pilot's  productivity  in  terms  of  output,  the  third  metric  is  the 
number  of  Current  Ship  Maintenance  Program  (CSMP)  work  items 
completed  by  the  Pearl  Harbor  NSY  &  IMF.  Although  this  metric  is  not  an 
indicator  of  the  former  shipyard's  productivity.  Navy  officials  have 
considered  the  number  of  CSMP  work  items  completed  by  their 
intermediate  maintenance  facilities  as  a  measurement  of  their 
productivity.®  The  CSMP  is  the  central  database  for  the  maintenance 
history  of  each  ship  and  contains  corrective,  preventative,  and  alteration 

®  Traditionally,  naval  shipyards  completed  a  minimal  number  of  CSMP  items. 
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maintenance  items  requiring  work.  The  Pearl  Harbor  NSY  &  IMF  estimated 
that  the  former  intermediate  maintenance  facility  completed  19,777  CSMP 
work  items  in  fiscal  year  1997.  However,  it  further  estimated  that  the 
consolidated  activity  completed  14,568  CSMP  work  items  during  the  fiscal 
year  1998  transition  and  7,786  work  items  as  of  May  in  fiscal  year  1999, 
indicating  no  improvement  in  performance  since  the  consolidation. 
Although  the  combined  facility  was  expected  to  maintain  the  same 
completion  rate  as  the  former  NIMF,  several  Navy  officials  believe  that  this 
expectation  was  unreasonable  because  the  number  of  military  enlisted 
personnel  decreased  from  1,275  in  October  1996  to  616  in  April  1999. 
Traditionally,  enlisted  personnel  completed  the  majority  of  CSMP  work 
items.  Consequently,  Navy  officials  have  discussed  lowering  the 
performance  expectation  for  this  metric. 

Strengths  and  Weaknesses  During  our  analysis,  we  found  that  this  metric  had  both  strengths  and 

weaknesses.  The  strengths  of  this  metric  are  that  the  fleet  and  ship 
commanders  maintain  the  baseline  and  test  data,  which  are  considered 
reliable,  and  the  NAS  plans  to  validate  the  baseline  and  pilot  results. 
However,  other  factors  not  related  to  the  pilot  affect  the  metric  that 
weaken  its  usefulness.  For  example.  Navy  officials  have  expressed  concern 
with  using  CSMP  work  items  to  measure  the  benefits  of  the  Pearl  Harbor 
Pilot  because  one  work  item  does  not  equal  another  item  in  terms  of  the 
labor  hours,  work  skills,  time  frame,  and  material  cost  required.  CSMP 
items  range  from  replacing  a  light  bulb  in  a  control  panel,  which  may  take  a 
worker  less  than  an  hour  to  replace,  to  overhauling  a  pump,  which  may 
take  several  workers  days  to  complete.  In  addition,  the  requirements 
needed  to  overhaul  one  pump  differ  from  other  overhauls  depending  on  the 
problem  and  the  type  of  pump.  Although  this  metric  has  weaknesses  that 
preclude  its  use  for  precise  measurements,  Navy  officials  believe  that  it 
provides  a  basis  for  an  order-of-magnitude  comparison. 


Current  Ship  Maintenance 
Program  Work  Items  in  the 
Backlog 


Historically,  Navy  officials  have  measured  the  material  condition  of  their 
ships  based  on  the  number  of  backlogged  CSMP  items.  Fewer  backlogged 
items  imply  that  the  ships  are  in  better  condition.  The  Navy's  performance 
expectation  for  the  combined  facility  was  to  reduce  the  estimated 
17,733-baseline  backlog  by  10  percent  to  15,960  work  items.  Recently,  the 
Pearl  Harbor  NSY  &  IMF  estimated  that  the  backlog  decreased  to  16,538 
items  as  of  May  1999.  Although  this  decrease  indicates  improvement,  it  falls 
short  of  the  Navy's  expectation  for  the  pilot.  However,  it  was  unlikely  that 
the  CSMP  backlog  would  decrease  significantly  during  this  period  because 
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of  the  reduction  in  the  number  of  military  enlisted  personnel  since  October 
1996. 

Strengths  and  Weaknesses  As  discussed  previous^,  using  CSMPs  as  a  test  metric  had  both  strengths 

and  weaknesses.  The  strengths  of  this  metric  are  that  it  traditionally  has 
been  considered  an  indicator  of  the  material  condition  of  fleet  ships  and 
that  the  fleet  and  ship  commanders  maintain  the  baseline  and  test  data, 
which  are  considered  reliable.  In  addition,  the  NAS  plans  to  validate  the 
baseline  and  pilot  results.  For  many  of  the  same  reasons  that  CSMP  work 
items  are  not  a  precise  measurement  of  output,  the  backlog  is  not  a  precise 
measurement  of  the  ship’s  material  condition,  which  makes  the  backlog  a 
weakness  in  regard  to  measurement.  For  example,  one  work  item  related 
to  the  safety  of  a  nuclear  submarine  could  potentially  restrict  the 
submarine  operations,  while  a  similar  item  on  a  surface  ship  could  have 
little  or  no  effect  on  its  operations.  Additionally,  other  factors  affecting  the 
backlog  include  non-pilot  related  items  such  as 

•  decommissions  of  PACFLT  ships  homeported  at  Pearl  Harbor,  which 
decrease  the  backlog  by  the  number  of  CSMP  items  recorded  for  the 
decommissioned  ship; 

•  visits  by  the  U.S.S.  McKee  (a  maintenance  tender),  which  decrease  the 
backlog  by  the  number  of  CSMP  items  completed  by  the  tender; 

•  maintenance  inspections,  which  increase  the  backlog  by  the  number  of 
unrecorded  CSMP  items  identified  by  the  inspection  team;  and 

•  procedural  changes  in  identifying  and  recording  CSMP  items,  which 
may  increase  or  decrease  the  backlog  depending  on  whether  the 
changes  weaken  or  strengthen  the  process. 

Although  this  metric  has  weaknesses  that  preclude  its  use  for  precise 
measurements.  Navy  officials  believe  that  it  provides  a  basis  for  an  order- 
of-magnitude  comparison. 


Schedule  Adherence  of  CNO  According  to  Navy  officials,  adherence  to  completing  CNO  maintenance 
Maintenance  Projects  projects  on  schedule  should  provide  an  indication  of  customer  satisfaction 

because,  from  the  fleet  and  ship  commanders'  perspective,  these 
maintenance  projects  should  be  completed  on  schedule. The  performance 


CNO  maintenance  projects  include  depot-level  maintenance  that  require  skills  or  facilities 
beyond  those  of  the  ship  and  intermediate  maintenance  activities  and  are  completed  by 
public  and  private  shipyards. 
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expectation  during  the  pilot  test  was  to  reduce  the  1 1.4  percent  schedule 
adherence  index  achieved  during  fiscal  year  1997,  since  a  lower  percentile 
would  indicate  that  projects  were  being  completed  closer  to  their 
scheduled  completion  dates.”  According  to  the  Pearl  Harbor  NSY  &  IMF, 
the  schedule  adherence  index  increased  to  18.6  percent  during  fiscal  year 
1998 — indicating  a  worsening  performance  during  the  transition  year. 
However,  Pearl  Harbor  NSY  &  IMF  data  showed  that  the  schedule 
adherence  index  for  the  CNO  maintenance  projects  completed  in  fiscal 
year  1999  has  exceeded  the  Navy's  performance  expectations  for  the  pilot. 
For  example,  although  the  first  CNO  project  completed  in  fiscal  year  1999 
was  20  percent  late,  the  four  subsequent  CNO  projects  have  been  either  on 
time  or  early,  decreasing  the  cumulative  index  to  3.7  percent  early. 

The  strengths  of  this  metric  are  that  it  traditionally  has  been  considered  an 
indicator  of  the  shipyard  customers'  satisfaction  and  that  NAVSEA 
maintains  the  baseline  and  test  data,  which  are  considered  reliable.  In 
addition,  the  NAS  plans  to  validate  the  baseline  and  pilot  results.  The 
weaknesses  include  that  the  metric  is  based  solely  on  the  former  shipyard's 
performance  in  meeting  the  completion  dates  for  CNO  maintenance 
projects  and,  consequently,  does  not  include  all  types  of  maintenance 
projects.  Pearl  Harbor  NSY  &  IMF  officials  believe  this  metric  should 
include  fleet  maintenance  projects  because  there  are  more  fleet 
maintenance  projects  than  CNO  projects,  thus  affecting  more  ship 
commanders.  However,  according  to  CNO  and  NAVSEA  officials,  they 
included  only  CNO  projects  for  this  metric  because  the  schedule  adherence 
data  for  fleet  maintenance  projects  were  considered  of  minimal  value.  For 
example,  the  former  intermediate  maintenance  process  was  to  defer  work 
rather  than  to  extend  the  duration  of  fleet  maintenance  projects  and, 
consequently,  these  projects  were  generally  completed  on  time.  Because 
the  pilot  has  not  changed  this  process  for  fleet  maintenance  projects,  CNO 
and  NAVSEA  officials  believe  an  analysis  of  schedule  adherence  data  for 
these  projects  would  be  of  minimal  value  in  assessing  the  pilot's 
effectiveness. 


"  The  schedule  adherence  index  is  the  sum  of  the  differences  in  the  actual  and  scheduled 
completion  dates  divided  by  the  total  scheduled  duration  (number  of  days)  for  each  CNO 
ship  maintenance  project  completed  during  the  fiscal  year. 
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Rework  Index  for  CNO  in  analyzing  this  metric,  we  found  strengths  and  weaknesses.  Measuring 

Maintenance  Projects  quality  of  work  completed  by  the  Pearl  Harbor  NSY  &  IMF,  the  sixth 

metric  is  determined  by  dividing  the  total  number  of  direct  rework  hours 
required  to  correct  work  deficiencies  by  the  number  of  direct  labor  hours 
delivered  for  CNO  maintenance  projects  completed  during  the  fiscal  year. 
Rework  hours  are  the  hours  spent  to  correct  work  deficiencies  after  the 
maintenance  project  is  completed. 

Strengths  and  Weaknesses  The  strengths  of  the  metric  are  that  it  traditionally  has  been  considered  an 

indicator  of  the  shipyard  quality  of  work  and  the  NAS  plans  to  validate  the 
baseline  and  test  data.  However,  its  weaknesses  include  that  the  baseline 
for  this  metric  is  limited  to  the  shipyard’s  CNO  projects  because  the  former 
intermediate  maintenance  facility  did  not  collect  rework  data  for  its 
projects.  In  addition,  the  metric  has  not  been  computed  for  fiscal  years 
1998  or  1999  because  no  one  has  assumed  responsibility  for  tracking  this 
information  since  the  inception  of  the  pilot. 

Activity  Work  Schedule  Measuring  the  pilot's  degree  of  schedule  integrity,  the  seventh  metric  is 

InteeritV  Index  calculated  by  dividing  the  budget  quantity  of  work  (direct  labor  hours) 

°  ^  scheduled  by  the  actual  amount  of  work  (direct  labor  hours)  delivered  to 

complete  the  maintenance  project.  According  to  the  test  plan,  actual  work 
exceeding  budgeted  work  indicates  inefficiencies  in  the  facility's  planning 
or  production  processes.  A  metric  value  of  more  than  one  indicates  the 
maintenance  project  was  completed  within  budget. 

Strengths  and  Weaknesses  The  strengths  of  this  metric  are  that  it  traditionally  has  been  considered  an 

indicator  of  the  shipyard  customers'  satisfaction  and  the  shipyard 
maintains  the  baseline  and  test  data,  which  are  considered  reliable. 
However,  one  weakness  of  metric  is  that  it  does  not  include  data  from  the 
former  intermediate  maintenance  facility.  In  addition.  Navy  officials  believe 
this  metric  is  similar  to  the  fifth  metric  measuring  schedule  adherence  and, 
therefore,  is  not  significantly  useful  in  determining  the  benefits  of  the  pilot. 
Consequently,  they  have  not  expended  resources  to  compute  the  metric 
value  for  the  baseline  year  or  established  performance  expectations  for  this 
metric.  Furthermore,  the  details  of  collecting  and  analyzing  the  data  have 
not  yet  been  determined.  According  to  CNO  and  NAVSEA  officials,  the 
Navy  does  not  plan  to  gather  the  data  needed  to  develop  and  analyze  this 
metric  unless  the  results  of  the  first  six  metrics  are  inconclusive.  For 
example,  if  the  pilot  results  for  those  six  metrics  are  mixed,  then  it  may  be 
necessary  to  measure  other  pilot  results,  such  as  this  metric. 
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Casualty  Reports  Caused  by 
Activity  Work 


Measuring  the  quality  of  work  completed  by  the  Pearl  Harbor  NSY  &  IMF, 
the  eighth  metric  is  based  on  analysis  of  casualty  reports.  These  reports  are 
filed  by  the  ship  after  any  equipment  failure  and  may  help  identify  work 
improperly  performed  by  the  Pearl  Harbor  NSY  &  IMF.  Those  reports 
indicating  improperly  performed  work  may  offer  insight  to  potential 
corrective  actions  to  the  facility's  methods  and  procedures. 


Strengths  and  Weaknesses  We  noted  strengths  and  weaknesses  in  our  analysis  of  this  metric.  The 

primary  strength  of  the  metric  is  that  the  fleet  and  ship  commanders 
maintain  the  baseline  and  test  data,  which  are  considered  reliable. 
However,  considered  a  weakness,  the  metric  does  not  include  data  from 
the  former  intermediate  maintenance  facility.  In  addition,  Navy  officials 
believe  this  metric  is  similar  to  the  sixth  metric  measuring  rework  items 
and,  therefore,  is  not  significantly  useful  in  identifying  the  benefits  of  the 
pilot.  Consequently,  they  have  not  expended  resources  to  determine  the 
details  of  collecting  and  analyzing  the  data  for  this  metric.  Similar  to  the 
previous  metric,  the  Navy  does  not  plan  to  gather  the  data  needed  to 
develop  and  analyze  this  metric  unless  the  results  of  the  first  six  metrics 
are  inconclusive. 


Earned  Value  For  the  earned  value  metric,  the  process  determines  the  value  of  a 

completed  task  by  comparing  actual  labor  hours  it  took  to  complete  a  unit 
of  ship  maintenance  work  in  fiscal  year  1997  with  actual  labor  hours  it  took 
to  complete  the  same  unit  of  work  in  fiscal  year  1999  and  subsequent  test 
years.  This  metric  attempts  to  measure  the  pilot’s  effect  on  maintenance 
outputs.  Based  on  an  analysis  of  the  former  shipyard  and  NIMF  production 
and  labor  data  for  fiscal  year  1997,  the  Anteon  Corporation  identified 
14  surface  ship  and  submarine  systems  that  were  considered  cost  drivers, 
consuming  the  most  labor  hours  during  the  fiscal  year.^^  On  the  basis  of  the 
distribution  of  labor  hours  for  each  of  these  cost  drivers,  the  Anteon 
Corporation  determined  that,  while  the  distributions  are  nearly  identical  in 
form,  they  were  not  distributed  normally  at  the  99.9  percent  confidence 
level.  The  most  common  statistics  used  to  describe  data  normally 
distributed  are  the  mean  and  standard  deviation.  In  this  case,  however,  the 
mean  and  standard  deviations  do  not  adequately  represent  the  data 
distribution.  Consequently,  non-parametric  statistics-which  are  generally 


The  Anteon  Corporation  defined  a  cost  driver  as  a  quantity  of  ship  maintenance  work  that 
represented  a  major  or  controlling  portion  of  work  on  a  particular  ship  maintenance  project. 
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Strengths  and  Weaknesses 


based  on  the  median  (as  opposed  to  the  mean)  -and  percentile  ranges  (as 
opposed  to  the  standard  deviation)  of  the  distribution  of  labor  hours  will  be 
used  to  describe  the  labor  hour  distribution  for  each  of  the  cost  drivers. 

If  the  distribution  of  labor  hours  for  the  14  cost  drivers  in  fiscal  year  1999  is 
less  than  the  fiscal  year  1997  baseline,  the  earned  value  metric  indicates 
that  the  productivity  of  the  work  performed  at  the  combined  facility  has 
improved  compared  to  when  the  facilities  operated  separately.  Conversely, 
if  distribution  of  labor  hours  in  fiscal  year  1999  is  greater  than  the  fiscal 
year  1997  baseline,  the  productivity  of  the  combined  facility  has  decreased 
compared  to  when  the  facilities  operated  separately.  Although  it  is 
expected  that  the  cost  drivers  from  fiscal  year  1997  will  be  similar  to  those 
in  fiscal  year  1999,  a  statistical  method  will  be  used  to  verify  that  the  data 
for  the  different  fiscal  years  are  similar.  If  the  data  are  not  statistically 
similar,  additional  analysis  of  data  will  be  conducted  to  develop  the  earned 
value  metric. 

A  strength  of  this  metric  is  that  it  attempts  to  measure  the  pilot's  effect  on 
maintenance  outputs.  However,  the  primary  weakness  of  the  metric, 
according  to  Navy  officials,  is  that  it  affects  such  a  small  population  that 
the  metric  is  not  significantly  useful  in  measuring  the  efficiency  of  the  pilot. 
Another  stated  weakness  is  that  the  metric  requires  significant  resources  to 
develop  the  results.  Consequently,  Navy  officials  have  not  expended 
resources  to  determine  the  details  of  collecting  and  analyzing  the  data  for 
the  metric.  Similar  to  the  previous  two  metrics,  the  Navy  does  not  plan  to 
gather  the  data  needed  to  develop  and  analyze  this  metric  unless  the  results 
of  the  first  six  metrics  are  inconclusive. 
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Appendix  III 
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Related  GAO  Products 


Navy  Ship  Maintenance:  Allocation  of  Ship  Maintenance  Work  in  the 
Norfolk,  Virginia,  Area  (GAO/NSIAD-99-54,  Feb.  24,  1999). 

Defense  Depot  Maintenance:  Public  and  Private  Sector  Workload 
Distribution  Reporting  Can  Be  Further  Improved  (GAO/NSIAD>98-175, 

July  23.  1998). 

Defense  Depot  Maintenance:  Contracting  Approaches  Should  Address 
Workload  Characteristics  (GAO/NSIAD~98-130,  June  15,  1998). 

Defense  Depot  Maintenance:  Use  of  Public-Private  Partnering 
Arrangements  (GAO/NSIAD-98-91,  May  7, 1998). 

Navy  Ship  Maintenance:  Temporary  Duty  Assignments  of  Temporarily 
Excess  Shipyard  Personnel  Are  Reasonable  (GAO/NSIAD-98-93,  Apr.  21, 
1998). 

Public-Private  Competitions:  DOD's  Additional  Support  for  Combining 
Depot  Workloads  Contains  Weaknesses  (GAO/NSIAD-98-143,  Apr.  17, 

1998). 

Defense  Depot  Maintenance:  DOD  Shifting  More  Workload  for  New 
Weapon  Systems  to  the  Private  Sector  (GAO/NSIAD-98-8,  Mar.  31, 1998). 

Public-Private  Competitions:  DOD's  Determination  to  Combine  Depot 
Workloads  Is  Not  Adequately  Supported  (GAO/NSIAD-98-76,  Jan.  20,  1998). 

Defense  Depot  Maintenance:  Information  on  Public  and  Private  Sector 
Workload  Allocations  (GAO/NSIAD-98-41,  Jan.  20,  1998). 

Navy  Regional  Maintenance:  Substantial  Opportunities  Exist  to  Build  on 
Infrastructure  Streamlining  Progress  (GAO/NSIAD-98-4,  Nov.  13, 1997). 

Navy  Depot  Maintenance:  Privatizing  Louisville  Operations  in  Place  Is  Not 
Cost-Effective  (GAO/NSIAD-97-52,  July  31, 1997). 

Defense  Depot  Maintenance:  Challenges  Facing  DOD  in  Managing  Working 
Capital  Funds  (GAOA'-NSIAD/AIMD-97-152,  May  7,  1997). 

Defense  Depot  Maintenance:  Uncertainties  and  Challenges  DOD  Faces  in 
Restructuring  Its  Depot  Maintenance  Program  (GAO/T-NSIAD-97-112, 

May  1,1997)  and  (GAO/T-NSIAD-97-111,  Mar.  18,  1997). 
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Related  GAO  Products 


Defense  Outsourcing:  Challenges  Facing  DOD  as  It  Attempts  to  Save 
Billions  in  Infrastructure  Costs  (GAO/T-NSIAD-97-1 10,  Mar.  12,  1997) . 

High-Risk  Series:  Defense  Infrastructure  (GAO/HR-97-7,  Feb.  1997). 


(709349) 


Page  44 


GAO/NSIAD-99-199  Depot  Maintenance 


